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PREFACE 



The idea for this conference arose through discussions and correspondence 
with Harry Kelly and Bowen Dees of the National Science Foundation. The conference 
was made possible through financial support from the National Science Foundation. 

In formalizing a proposal and in contemplating the functioning of the conference, 
it was recommended that a steering committee be formed. Fortunately, J. P. Guilford, 
Benjamin S. Bloom, and Donald W. Taylor all accepted the invitation to serve on the 
steering committee. All three were extremely helpful in planning and organizing the 
conference and should be credited for numerous contributions which made the confer- 
ence function more smoothly and effectively. Due to an unexpected move from Califor- 
nia to a new position in the east, Donald W. Taylor was unable to attend and report his 
research work after playing a strong role in the organization of the conference . 

Many local persons rendered invaluable assistance in the planning and function- 
ing of the conference as well as in the arduous tasks of transcribing and preparing this 
final report. William R. Smith and C. Larry Hagen helped with the planning and effi- 
ciently carried the entire burden during the period of my required military tour of duty 
shortly before the conference convened. William R. Smith, JohnR. Cochran, Boyd 
Sheets, Jeane Stillman, and others were responsible for various activities during the 
conference, such as transporting all participants between the airport and the lodge high 
in the Wasatch Mountains at Brighton, obtaining excellent complete transcriptions of all 
reports and total -group discussions during* the conference, and many other ’’behind -the- 
scene” activities which freed the participants to focus practically their entire attention 
upon the conference topic and discussions. The Alpine Rose Lodge, in its secluded, 
scenic setting, proved to be an excellent location for a research conference . 

The participants whose names are listed in the Table of Contents deserve many 
thanks for their reports and their many lively and friendly exchanges; in reality, the 
outcome of the conference hinges primarily upon the contributions of the participants . 
They brought a rich background of scientific training and research experience to bear 
upon the difficult problem of creative scientific talent. They also had already found a 
wide variety of approaches to this complex problem. 

The conference was scheduled just prior to the 1955 annual meeting of the Amer- 
ican Psychological Association, in order to permit participants from the east to stop 
over en route to the APA meetings in San Francisco. Unfortunately, a small number 
of highly qualified persons who were invited to participate were unable to attend due to 
conflicting engagements or illness . 

In research conferences of this type in which there is an ’’inner circle” of par- 
ticipants it seems wise to make available, to as wide an audience as might be interested, 
the essential ideas expressed during the conference. A serious difficulty arises when 
any single individual attempts to judge from his subjective frame of reference, the future 
worth of every idea expressed. This becomes necessary whenever it is decided to pre- 
pare summary reports, containing ’’only those ideas which are most valuable. ” It may 



be that some of the best ideas in research conferences are strange, fresh, and unortho 
dox to many current researchers who might tend to feel more comfortable and be more 
efficient if they were to extract and report the more orthodox ideas and approaches . In 
order to avoid possible biases of reporting, we decided with strong conviction to pub- 
lish a practically complete final report, if this could possibly be done. It is satisfying 
to know through all the vital assistance described herein that this trial attempt at full 
reporting has now been accomplished. It is hoped that not only the ideas expressed but 
also the spirit of search so fully displayed in the conference, plus the occasional tenta- 
tiveness of early thinking on certain topics, have been adequately communicated to the 
reader. 



Some speakers presented their reports by reading or following a prepared paper 
rather closely; in such cases, transcription work was not needed except where there 
were major changes in expressions and where other participants entered into the dis- 
cussion. Other persons preferred to talk from notes. The discussion portions of the 
conference were generally spontaneous and free-flowing. 

Oyer twenty hours of rape recordings were obtained during the three days of 
meetings. Carolea Orton and Grettle Hansen deserve tremendous credit for their pains- 
taking, successful efforts in understanding and transcribing these recordings; in fact, 
it was necessary cO re-run so many portions of the tape that a reversing part of the re- 
corder was broken from excess wear. 

After the participants had edited their own papers and discussion pertaining to 
them, the total report was assembled into one consistent format through the efforts of 
Philip Sturges and Hazel Myers, who also provided a final, thorough proofreading of 
the multilith masters. Joyce Stillman receives foil credit for the typing of this final 
conference report - -her productivity and skill on the electric typewriter are highly re- 
garded by all who have worked with her on this and other research projects. 

An amusing experience happened to me during the transcription process . After 
all reports had been transcribed, I asked for the transcript of my report. No one could 
find it or even remember making a transcription, so we replayed the tape and verified 
that there was no evidence on the tape that I had given a report. We then recalled our 
concern about the possibility of running out of tape. I had apparently signaled to those 
operating the recorder to skip my report so that we would be sure to record all further 
comments made by our visiting participants. Being the only one not provided with a 
draft of his report, I realized I would be forced to rely upon my notes, but unfortunately 
these notes were misplaced during a move of our research projects and have not yet 
been found. If these notes ever reappear, it will be interesting to compare them with 
the report I have written herein. 

Discussion on a report has generally been presented where it actually occurred, 
whether it was during the course of the report or at the end. Three symbols have been 
utilized to indicate exchanges when the discussion changes from one person to another. 
The symbols used at the beginning of a paragraph whenever a different person expresses 
himself are listed on the following page . 



S--The speaker who is giving the report is doing the talking in this and in suc- 
ceeding paragraphs until a different symbol appears at the beginning of a new paragraph. 

C--A participant, otherwise unidentified except that he is not the present speaker, 
is making a comment. 

Q.—A participant, otherwise unidentified except that he is not the present speaker, 
is raising a question. 

There were two main reasons for not identifying the participant in each exchange. 
First, it would have been a large extra burden to have attempted to identify each partici- 
pant’s exchanges to any high degree of accuracy. Second, it is believed that any good 
ideas in science must ultimately be able to stand on their own merit independent of wheth- 
er they came from an authoritative figure or from a relatively unknown person. 

It was evident, in retrospect, that the attempts to discuss and identify the crea- 
tive type of scientific talent would be considerably simplified if a clear picture of the 
nature of scientific talent, in general, were already available. At times during the con- 
ference it seemed as if the attention was centered on the nature of scientific talent with 
the hope that when this was clarified, it would be easier to single out and contrast the 
creative type of scientific talent. 

In summary, the staging of a conference of this type plus the preparing of an al- 
most verbatim final report cannot be accomplished without valuable contributions and 
cooperation from many people. It has also been a pleasant experience to have worked 
with every one of these persons . The responsibility for any shortcomings in this ven- 
ture belongs to me alone. We hope that the final report, as well as the conference, will 
have the effect of stimulating increased research in this challenging area. 



August 17, 1956 



Calvin W. Taylor 



GREETINGS TO PARTICIPANTS 

President A. Ray Olpin 
University of Utah 



S--It is an honor to have a part in the promotion or fostering of a conference of 
this kind. Before I came to the University of Utah my chief interest wa 3 research, and 
I had considerable experience in the management of research laboratories. My concern 
for creative endeavor continues. I recognize the scarcity of highly qualified research 
personnel and the expanding need for individuals properly trained for scientific investiga- 
tion. I think any effort directed toward identification of young people with creative 

ability - -nonconformists, if you prefer to call them that -- is a challenging subject for 
all of us . 



At noon today I met with a group of people from all parts of the nation who have 
assembled on the University of Utah campus in a convention of the National Association 
of Children's Theaters. Every speaker at this convention dealt with the development 
of imagination and creative ability in young people, even tiny youngsters. They seem 
to be engaged in the promotion of the kindergarten stage of the problem we are concerned 
with tonight. My brief remarks to the Association were in the nature of commendation 
of their efforts to encourage young people in creative expression. 



We are assembled here now to discuss ways and means of discovering talented 
youth in all fields of learning — the arts, the humanities, and in all scientific and pro- 
fessional fields. We are living in a population which is made up predominately of young 
people. The accent today is on youth, as was so dramatically stated by the late Thomas 
Midgley, then vice president of General Motors, Ethyl Corporation, Dow Chemical, and 
various other organizations. He was speaking as the president and chairman of the 
Board of the American Chemical Society. I was particularly interested in his comments, 

^ he was in a sense my boss, as vice president of the Research Foundation where I 
was director. 



Midgley was in a wheel chair when he delivered this timely speech entitled 
"Accent on Youth, " for at age fifty he had contracted poliomyelitis and as a result, 
suffered complete paralysis of the lower part of his body. Even with this infirmitry, 
however, he contributed greatly to the winning of World War II. He was not a defeat- 
ist. He looked ahead and saw in the future great potential. His spirit and attitude were 
always an inspiration to me. Perhaps I could give you a little insist into the attitudes 
and philosophy of the man by referring to a letter I received from him shortly after he 
was stricken with paralysis while he was still in the Children's Hospital in Columbus. 
This letter was in response to one I had written him enquiring about the seriousness of 
his affliction. He wrote as follows, under date of October 31, 1940: 
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Dear Mr. Olpin: 



Thanks for your letter of October 2nd. 

I have been spending my spare moments in bed figuring out the 
statistical probabilities s of a fifty -one year old male catching polio- 
myelitis, as I have, and this comes out to be substantially equal to 
the chances of drawing a certain individual card from a stack of play- 
ing cards as high as the Empire State Building. It was my tough luck 
to draw it, but on the other hand, I have figured out that it is fifteen 
thousand times more likely for a n?*ile aged fifty-one to have a cor- 
onary thrombosis, or twelve thousand five hundred times more likely 
to have an incurable cancer, or seven thousand five hundred sixty- two 
times more likely to be bumped off in an automobile wreck. All of 
which can be taken to prove how lucky I really am. 

As you no doubt recognize, the above is a lot of damned silly 
Polly Ann rot. I am unlucky in getting this vile distemper, but it 
has its real compensations in discovering what a splendid bunch of 
friends I have who have written me as you have. This helps a lot. 

Best regards, 

/s/ Thomas Midgley, Jr. 

There you have a reflection of the make-up of a man with a creative talent, a 
man with inventive genius. It was this person who invented tetraethyl lead by combin- 
ing certain chemicals which, when poured in small quantities into gasoline for the pro- 
pulsion of combustion engines, gave to this country over night added power equivalent 
* t0 eighty times that generated by Boulder Dam. It was this same scientist and engineer 
who invented Freon, one of our safest refrigerants, by combining perhaps the most toxic 
element known, flourine, with one of the most combustible gases, methane. 



Yes, Thomas Midgley had a way of piercing the unknown and coming up with new 
relationships that was unique. He was optimistic about the future because he had con- 
fidence in youth. Although he was only about fifty -four when he died, I felt he had done 
a lot more living during a half cpntury than many people who were much older than he. 
He had been trained as a mechanical engineer, yet he lived to win practically every 
award given for outstanding contributions in chemistry. He felt the need for placing 
young men in responsible positions and illustrated his point by reference to major con- 
tributions made by them in all fields of endeavor. He thumbed through the pages of 
history to impress on his audience the remarkable youth of a few great commanders, 
pointing out that Alexander had conquered the world at twenty -five, Hannibal was only 
twenty -six when he became commander -in -chief of the armies of Garthage, Napoleon 
proved himself the greatest military genius of our modern world at twenty-seven; 
Caesar alone of such men, Midgley discovered, was forty before his genius became 
outstanding. After citing these men of history, he wondered if we might not have been 
better off in the last war had we chosen younger men for high command; and here he 
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was unable to draw a clear conclusion for he observed that Napoleon was ultimately de- 
feated by his senior, Hannibal with all his genius ended in defeat and the extermination 
of his nation, Caesar was assassinated, leaving only Alexander to die in honor and by 
natural causes. He, said Midgley, drank himself out at thirty-three. These comments 
were followed by reference to some older military strategists, leading lip to this con- 
clusion, "For genius and display, take youth; for cold calculation and planned execution, 
take age. In either case, choose wisely. ” 

Continuing, Midgley noted the youthfulness of genius in such nonrelated fields as 
mathematics and music. Pascal at sixteen wrote of conic sections. Laplace was inter- 
nationally recognized before he was twenty-one. Gauss was well known in his teens for 
his mathematical researches. Galileo developed the laws of the pendulum when only 
eighteen. Mozart composed at five and conducted the opera at Milan when only fourteen; 
Beethoven was court organist at the same age. 

Because these youthful geniuses might be considered biological abnormalities, 
Midgley looked for a more convincing argument that creative ability comes at an early 
age. He went to the records of the United States Patent Office and came up with a table 
of important inventions listing the inventor and his age . It starts with a young man, 

Sir William Perkin, aged eighteen when he invented aniline dye. Then follow names of 
the persons who invented steam engines, governors, electroplating lead with copper, 
the revolver, wireless,, airbrakes, the first reaper, reduction of aluminum, stock 
ticker, spinning jenny,, sewing machine, cotton gin, steam fire engines, etc., inven- 
tions which have influenced our lives right down to the present. Very few of these dis- 
coveries were made by persons over forty -five years of age. This, he continued, is 
striking evidence that young people are the nonconformists, the ones who originate new 
ideas. For that reason he felt that older people should step aside by retiring early in life 
to permit younger people to move into positions offering opportunities to create. The older 
people can be used in administrative posts where experience is valuable, he concluded. 

It was my close association with Tom Midgley and my administration for him and 
his clear thinking that convinced me of the importance of identifying creative talent in 
people when they are young and setting up procedures for accelerating their develop- 
ment of this creative ability. We are promoting the use of group dynamics, aptitude 
tests, personal interviews, and every technique now known to detect those individuals 
who show most promise as they move ahead in their academic pursuits. We realize, 
however, that many of our most brilliant young people fail to reach college, and that 
many with creative talent are not sufficiently challenged at an early time in their col- 
lege careers and may, therefore, lose interest and drop out of school. It is very grat- 
ifying to me, as president of the University of Utah, to sit down with you people who 
have been assembled here under Dr. Taylor's leadership to concentrate on this problem. 
I am certain that as a result of your deliberations valuable suggestions will be advanced 
and perhaps methodologies developed to assist us in finding, stimulating, and acceler- 
ating the development of young people with creative talents and interests that we may 
better serve our society by developing men of leadership qualified to maintain and ex- 
pand the economy of this great country of ours. 

I hope you enjoy your stay here. I know you will find the cool nights at this moun- 
tain resort invigorating and the rugged mountain scenery inspiring. We invite you, 



howeyer, to schedule some time during your stay in Utah to visit the main University of 
Utah campus m Salt Lake City. We trust we may become better acquainted with you and 

you with us so that you will want to return often. We are honored with your presence, 
and I wish you well in your mission. * P 



NSF'INTERESTS IN THE PROBLEM OF IDENTIFYING CREATIVE SCIENTIFIC TALENT 

* 

Bowen C. Dees 
National Science Foundation 



S— My first and most pleasant duty is to convey to you, President Olpin, to Dr. 
Taylor, and to other members of this conference the greetings of the staff of the Na- 
tional Science Foundation; and to pass along from all of those who have been interested 

in this conference at the Foundation the hope that this will be a most successful confer- 
ence. • ; 



It would be nice if I could assume at the outset that all of those who are present 
here are thoroughly familiar with the history and the functions of the National Science 
Foundation. Four years of experience in discussing NSF interests with people through- 
out the country have made it clear to me that a certain amount of explanatory material 
on what the Foundation is and what it is trying to do is generally helpful in situations of 
this kind. Let me take just a few moments, therefore, to review what the foundation is 
trying to do, which for some of you will undoubtedly be old material. 

The Foundation was established by Act of Congress in May 1950. By the end of 
that year the twenty-four -member National Science Board spec *ied in the statute had 
been appointed by the President. By April of 1951, Dr. Alan Waterman hadbeen chosen 
to serve as Director of the Foundation; and shortly thereafter he began to associate 
with himself the staff required in order to begin operating the new independent agency. 

Among the several functions specified as the responsibilities of the National 
Science Foundation are two that are of particular concern in connection with this meet- 
ing. The first of these has to do with the Foundation's program for the support of basic 
research. Through the mechanism of grants to individual investigators throughout the 
country, the Foundation has for some four years now been helping to advance the fron- 
tiers of scientific knowledge. The second program - -and the one which may prove to 
be of major concern to this group - - is the Foundation’s fellowship program, through 
which it extends financial aid to large numbers of graduate students and somewhat 
smaller numbers of postdoctoral workers to enable them to continue their training in 
science. The first NSF fellowship awards were made in April 1952, and the first group 
of Fellows began to work under their fellowships in the summer and fall of that year. 

The fellowship program is the activity of the Foundation which is most immedi- 
ately and specifically concerned with the topic which is central to this conference. At 
the present time some 800. awards are made each yeai to predoctoral and postdoctoral 
Fallows . A program of this magnitude, involving as it does the expenditure of approxi- 
mately 2. 3 million dollars annually, requires continuing analysis in an effort to assess 
its past contributions and to improve its future operations. One of the things which has 
been of continuing concern to us is the possibility of improving the selection techniques 
utilized in choosing recipients of NSF fellowships. 



We are frequently asked what it is that the program is attempting to do.. Many 
people have indicated to us that it is virtually impossible to suggest ways of improving 
the Foundation's fellowship selection techniques, unless we can state what it is we are 
selecting Fellows for in rather specific terms. It is difficult for us to do this in a simple 
way. The fellowship program of the Foundation extends over all of the fields of science 
and is applicable to individuals at all levels of graduate study. The fact that we have in 
the program engineers as well as abstract mathematicians, students in the medical 
sciences as well as students in nuclear physics, means that it is difficult to specify in 
general terms exactly what it is that we want to have as the outcome of these awards. 

In the case of a theoretical physicist the aim is most likely to be the production of a 
research worker who can make new and significant contributions to theoretical physics. 
In summing up these aims perhaps the best way of saying what it is we wish to accom- 
plish is to say simply that we hope and expect that the program will result in strength- 
ening the scientific and technical leadership of the United States. We are, in other 
words, seeking potential scientific leaders for these awards. 

It is not necessarily true, as we see it, that leaders must in every case be inno- 
vators. It is certainly conceivable that leadership in an administrative post involves 
a different kind of ability from that required to produce new and outstanding findings in 
abstract mathematics. Yet, it is obvious (and this is the reason for this conference 
being supported by the National Science Foundation) that we are intensely and continu- 
ingly interested in the question of finding and nurturing those individuals who possess 
the potentiality for creative accomplishment in science. 

On June 24 and 25, 1954, the Foundation brought together in Washington, a group 
of individuals to take part in a conference on "Techniques of Selecting Recipients of 
Fellowships. " The purpose of that conference was essentially this: By bringing to a 
focus the thoughts of individuals from three areas --scientific investigation, psychom- 
etry, and fellowship program administration - -we hoped to obtain a critique of the 
selection techniques currently being used, so that we could be sure that these techniques 
are satisfactory --or, alternatively, so that we could begin immediately to institute 
improvements . Included among the topics on the agenda for that conference was one 
which read as follows: "How should the concept of creativity of science be dealt with 
in connection with a fellowship selection process?" In a working paper which accom- 
panied the agenda we spelled out in brief what seemed to us to be of concern in this 
area. I would like to quote the pertinent paragraphs from that document: 

"What distinguishes a highly original scientific investigator from 
an apparently equally intelligent scientific contemporary who simply 
develops or analyzes ideas of others? How can the term creativity 
in science be defined meaningfully? Is it possible to develop measures 
of potential for creativity in science and, if so, at what stage in an 
individual’s development can such measures be applied? What addi- 
tional research is needed in this field? 

'The foregoing questions are only a few of those which could be 
raised in connection with the general problem of finding and nurturing 
those students who have high potential for becoming original, creative 
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research workers in science. The problem of creativity is obviously 
one which goes far beyond the scientific disciplines. It is a field in 
which much reseat ch is currently proceeding, but there seem to be 
few if any instruments available at this time which would be of direct 
usefulness in a fellowship selection operation. 

"Agenda item C (the one quoted just above) is not intended to elicit 
detailed discussion of the research which is going on in this field or 
in potential findings, but rather to focus attention on this area in an 
attempt to get (a) points of view from the participants as to the im- 
portance of finding' ways of measuring creativity, and (b) subjective 
judgments as to the possibility of success in research efforts designed 
to produce predictors of future creativeness in science. As has been 
pointed out in the note at the end of the agenda, an additional session 
or sessions may be provided after the formal end of the conference 
for additional discussions on this subject if there are enough inter- 
ested participants to make such sessions desirable. " 

The idea of a post -conference session on creativity appealed to several partic- 
ipants. On the afternoon of the second day, therefore, we discussed for about two 
hours the general question of creativity - - what might be done about trying to learn 
something more concerning the work currently under way in this field, and what new 
steps might be taken. There was general agreement that a much more detailed dis- 
cussion on these matters would be desirable, and the suggestion was made that a con- 
ference such as the one we are now initiating should be called at the first opportunity. 

As I imagine many of you know. Dr. Calvin Taylor was for two years Director of Re- 
search in the Office of Scientific Personnel at the National Research Council, and had 
as his major responsibility there the carrying out of research projects having to do 
with the National Science Foundation fellowship selection process. In view of his in- 
terest in and knowledge of our specific problems we felt that it was particularly ap- 
propriate for the Foundation to consider supporting his proposal for this conference 
when it was submitted by the University of Utah. 

I have said a good bit about the Foundation’s interest in the question of identifying 
creative scientific talent from the standpoint of its fellowship program. Obviously, it 
is important to be in a position to identify such talent in connection with the operation 
of the Foundation’s research grants program. Therefore, the other divisions of the 
Foundation are also interested in the outcome of this conference. Their interest is some- 
what different from that of the fellowship group, since they are primarily concerned 
with determining the potential contributions of senior investigators who apply to the 
Foundation for support of specific research projects. Their interest is nevertheless 
a very real one, and it is highly probable that the results of this conference will be 
found of value to their programs as well as to the fellowship program. 

In addition to the interest of the research grants group from a purely adminis- 
trative point of view, however, there is another facet of their interests; Dr. John 
Wilson, as Program Director for Psychobiology, is particularly interested in the out- 
come of this conference and was very unhappy when it became obvious that he would 
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not be able to attend these sessions . His interest stems from the fact that there are 
very likely to be ideas developed at this meeting which will bear directly upon the 
research -supporting program which he administers. Any basic research in the area 
of psychology which may result from or be expanded by virtue of these discussions will 
be of concern to him, both professionally and in terms of his program responsibilities 
at the Foundation. 



There is still another facet of the creativity problem which is sure to receive 
some attention here and which the Foundation will wish to consider with interest. This 
has to do with the fact that there are almost surely environmental conditions which ex- 
ert an important influence on the creativity of specific individuals. The Foundation is 
charged with responsibilities in the general area of developing and encouraging the 
pursuit of a national policy for scientific research and education. Therefore, it is 
obvious that the Foundation would be interested in any research findings that shed new 
light on ways of improving the scientific output of the United States through adopting 
administrative or other environment-modifying procedures conducive to better scien- 
tific output- -either in quantity or in quality. 

The preamble of the. National Science Foundation Act of 1950 states that this 
piece of legislation is ”AN ACT To promote the progress of science; to advance the 
national health, prosperity, and welfare; to secure the national defense; and for other 
purposes. " Even before the first legislation which led to the establishment of the 
Foundation was beamed, there was much concern over the fact that this nation had been 
failing in a serious way to make best use of its potential scientific brainpower. In 
Science- -the Endless Frontier , Vannevar Bush’s famous report which gave considerable 
impetus to the move toward establishment of a new national scientific agency, a great 
deal of attention was devoted to questions of scientific manpower. That report, pub- 
lished in 1945, quoted a statement by Dr. Conant which is as applicable now as it was 
then: 



”... in every section of the entire area where the word science may 
properly be applied, the limiting factor is a human one. We shall have 
rapid or slow advance in this direction or in that depending on the num- 
ber of really first-class men who are engaged in the work in question. 

. . .So in the last analysis, the future of science in this country will be 
determined by our basic educational policy. ” 

Dr. Bush went on to point out that World War II had, in his opinion, ”... drawn 
too heavily for non- scientific purposes upon the great natural resource which resides 
in our trained young scientists and engineers. For the general good of the country 
too many such men have gone into uniform, and their talents have not always been 
fully utilized. . . . (During World War II) those ready for college training in the sciences 
have not been permitted to enter upon that training. . . .There is thus an accumulating 
deficit of trained research personnel which will continue for many years. . . .With 
mounting demands for scientists both for teaching and for research, we will enter the 
post-war period with a serious deficit in our trained scientific personnel.” 



The concern expressed over the country's scientific manpower supply in the Bush 
report has been a continuing one over the last decade. Due in large measure to the up- 
surge in college enrollment made possible by the GI Bill of Rights, we have seen a level 
of production of scientists and engineers reached in the post-war years which exceeded 
by a significant percentage the level projected at the end of the war. These production 
figures have, however, been matched by a similar increase in demand for scientific 
and technical personnel, and the cry "We must train more scientists and engineers!" 
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One of the statutory functions of the National Science Foundation is the mainten- 
ance of a clearinghouse of information concerning scientific and technical personnel. 

In connection with this responsibility, the Foundation has attempted to obtain reliable 
figures concerning the current supply -demand situation relative to scientific and tech- 
nical personnel. This is not an easy problem. The evidence is clear that the univer- 
sities, industry and government sometimes have difficulty in hiring all the scientists 
and engineers, trained in certain specialties, that they need or want. In some fields 
of science, however, there appear to be adequate numbers of trained individuals avail- 
able, and indeed in a few specialties we have heard that recent Ph.D. *s have difficulty 
obtaining positions worthy of their degree of training. Recognizing the difficulty of 
obtaining accurate estimates of shortages, field by field, and not wishing to put itself 
in the position of "crying wolf" unjustifiably, the Foundation has exercised consider- 
able care in making statements relative to the so-called "shortage" problem. I have 
with me copies of our Scientific Manpower Bulletin No. 6, which deals with a recent 
study done for the Foundation by the Bureau of Labor Statistics. Although this re- 
port is largely non-quantitative in nature, it brings out several significant aspects of 
the present supply-demand situation in some of our major industries. Shortages 
exist, this report indicates, but not in all industries and not in all fields. Although 
we are not ready to join those 1 who are currently calling for more, more, and still 
more scientists, two things stand out in our overall analysis of the present situation: 

1 . A real crisis lies ahead of us in science education due to the fact that 
entirely too few well -prepared high school teachers of science will be 
available to teach the large numbers of youngsters now beginning to 
swell the ranks of our high school classes; this teacher shortage is an 
area where projections of needs can be made with a fairly high degree of 
accuracy, and where we, therefore, are in a position to say with some 
conviction that a real and very serious problem exists. 

2. Whatever the situation may be relative to the numbers of scientists need- 
ed over the next decade or so, we can be certain of this: we need to im- 
prove the quality of science education and to improve our techniques for 
bringing into science those who are most capable of becoming leaders 

in the field. 

Few discussions of this kind these days can be expected to omit reference to 
Russian scientific manpower developments. Information which seems to be quite 
reliable indicates that the Russians are accelerating their education and training 
programs in scientific and engineering fields. Many informed observers are convinced 
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that the USSR is even now approaching comparability with the United States in both quan- 
tity and quality of scientists and engineers. The meaning of these facts relative to 
Russia is interpreted in various ways. Some feel that the United States should immedi- 
ately take steps to induce - -and possibly even coerce -- more students into the study of 
science and engineering because Russia has done so. It is our conviction that this approach 
is in error. The United States should obviously train as many scientists and engineers 
as are needed to advance its science and technology at an optimum rate. However, we 
might lose more than we would gain if we were to allow our fear of what Russia is doing 
in this field to stampede us into actions which are inconsistent with our traditions and with 
our economic and scientific requirements. 

Obtaining more knowledge of the nation’s current needs and potential resources in 
the field of scientific manpower, establishing more programs to improve science educa- 
tion, finding better ways of identifying — and motivating toward careers in teaching or 
research --those individuals who show scientific promise; these and other important chal- 
lenges in the area of science education face us all, and bulk large in the National Science 
Foundation's current efforts to ascertain and fulfill its proper role. 

It is clear from the agenda with which Dr. Taylor has provided us that one of the 
possible outcomes of this conference will be a proposal or set of proposals having to 
do with additional or expanded research efforts in the area of identification of creative 
talent. From what I have already said, I hope it is clear that the Foundation will have 
a very real interest in these proposals and will be happy to consider supporting some 
or all of them. A limited amount of research directed toward specific elements of the 
fellowship selection problem can be supported by research funds allocated to the Pro- 
gram Director for Fellowships. Research of this kind would probably be classified as 
applied, rather than as basic research, although this is not necessarily the case. Pro- 
posals in the basic research area will be of interest to all of us, but action on these 
would in all likelihood be assigned to Dr. Wilson as Program Director for Psychology. 

I am delighted to be here with you for these few days, and I am sure that the ex- 
perience will be a rewarding one for me. Since my background is in physics, I would 
suppose that much of what will be said will be completely outside the range of my limited 
knowledge of psychology. After several years of exposure to conferences of this kind, 
however, I have learned that by listening and maintaining a discreet silence I can profit 
considerably by participating in such meetings . I had originally intended to limit my role 
here to that of an observer, but Dr. Taylor persuaded me that a background statement as 
to the Foundation’s interest in this field might be helpful. You may have some questions 
on matters which I have mentioned or on others dealing with the Foundation and its work; 
if so, I will be delighted to try during the next few days to answer them. For the moment, 
however, I shall consider that I have earned the right to relax and step into the role of an 
interested observer. 

Q- -You referred to some areas of science and engineering in which, at the present 
time, there is no shortage. Would you mind stating those? 

S--In some of the life sciences there appear to be cases where individuals simply 
can't get jobs. Human genetics is one such field; a very good man who graduated last 



year from one of our good universities found that he just could not get a job in this field. 

A man in a field close to comparative anatomy has just taken another job, moving from 
a place where he had a very miserable salary: but he now won’t be in his field. So 
there are. cases of this kind, where the shortages are not nearly as great as one would 
imply if he simply said, ’’All scientists are in great demand. ” I might add by the way 
that this is true as far as our studies show in some scattered areas of physical science 
and engineering also. Certainly there are differential demands in engineering even, and 
in the physical sciences as well. 

-Dp you know of any place in the country where steps are being taken in any way 
to alleviate the impending shortage of high school science teachers? 

S--I know of no effort of anything like the magnitude that will be required to solve 
this problem. This is such a critical and monumental problem that we are tearing our 
hair out to find promising ways of tackling it. 

Q- -Do you have any leads worth thinking about? 

S — Yes, a few. Henry Chauncey of ETS, for example, has been pushing the notion 
of science film courses— not just audio-visual aids, but complete science courses on 
film- -which would present the subject matter of physics or chemistry or algebra 
whether or not a qualified teacher was available. This sort of thing practically always 
hits a stone wall when you are talking with educators or scientists, because they say: 

’’You can’t do without the personal contact; you have got to have a well -qualified teacher. ” 
The problem that many of these people have not yet faced is that we are not going to have 
these teachers. The facts of life in this. situation are extremely clear. In order to staff 
our public high schools and elementary schools at the current student- teacher level, half 
of all college graduates the next decade would have to enter public school teaching. Only 
about 20 percent, at most, of our college graduates today are going into teaching. To 
think of a 250 percent increase is completely unrealistic. 

Q- -We have some public school people here from this state and you are probably 
familiar with the others. But when the American Institute of Physics was established in 
1936, I happened to be one of the speakers at the Hotel Pennsylvania on the training of 
physicists for industry. Someone made the statement then that there are states in the 
Union that prohibited, by law, the teaching of physics in the high schools . I wonder if 
there are any such places at the present time, and are there any places where they are 
tightening the requirements in states or regions in the high schools in the fields of math- 
ematics, physics, and other scie ices? When we get them at the college level we some- 
times wish that more of them had had basic training, and I wondered if states were 
doing anything to tighten their requirements? 

S--With respect to your first question, there are no states now that have laws 
against the teaching of physics because it is taught in all states. However, as one 
element of an answer to the second part of your question, there are a number of states 
where physics is taught in no more than half of the high schools of the state. Over the 
last few years, even with high levels of high school class size, the percentage of 



students and even the absolute numbers of students taking physics have been going down* 
In biology the absolute number has been rising and the percentage has risen slightly. In 
algebra, however, the percentage of students who have been taking algebra has been 
going down and the percentage taking one or another of the general mathematics courses 
has been going up. (These are the courses in business arithmetic or consumer arith- 
metic --all kinds of terms are used to describe these generalized courses. But the fact 
remains that they are not good courses in preparing any student for collegiate work. ) 

Standards for graduation from high school, as we understand them, vary widely 
over the country as a whole. It is possible for students to be admitted to the state uni- 
versity or to reputable schools in some states with only one unit of mathematics in their 
high schodl course. There has been a hue and cry for a long time to this effect: "Don't 
insist that a student make up his mind tod soon.. Give him freedom of choice when he 
gets to the university or the college to choose what path he wants to follow as a college 
student. " I think that scientists wouid be very reprehensible if they were In any way to 
try to deny the validity of this position. But the point remains— and it has too infre- 
quently been made- -that students who get out of high school with no science and with 
little or no mathematics do not have freedom of choice to choose science as a career 
when they enter college. I have been trying to convince as many people as possible 
that, if this "freedom of choice" is to be a real thing for our youngsters, we should see 
to it that they have enough math and science in high school so that they can enter science 
if they want to when they go to college. 

Financially speaking, a student of education has never been among the happier 
lot. As a matter of fact, many of them compromise on a profession by going into teach- 
ing because they are not able to afford longer programs in dentistry, medicine, or ex- 
tended Fh.D. programs. Have you folks ever considered at the National Science Foun- 
dation the possibility of priming the pump with limited fellowships or small grants ihat 
might be granted to students in education to channelize them into science where they 
might be at a loss as to where they might want to teach if at all? 

S - -Are you thinking of the under -graduate level primarily? 

C— You will probably have to come in some time in the upper -division years, or 
in those states where a fifth year is required, it might come up there. 

e 

S--We have certainly considered this and have been swayed in our thinking on this 
by virtue of the fact that, as of the moment at least, the general problem of undergrade 
uate support just seems to be a little too big for us to get into. It's a tremendous pro- 
blem once you get talking about the numbers that could be involved, and unless you talk 
about big numbers then you ’are not really getting at the problem because the problem is 
so large. At the graduate level, that is, the post -baccalaureate level, we have been 
nibbling at the edges of this problem some and in recent months have been toying with 
the notion of various support programs that might help to get more people into teaching. 
The overall problem, however, does not look promising in terms of those statistics I 
quoted earlier. No niatter what we hope for in terms of a realistic fraction of college 
graduates going into high school and elementary school teaching, we're not going to 
have enough. This is the sum total of the thing. 



There are other means, other than this one thing on teaching films that I men- 
tioned, that are being tried in this connection: the notion of bringing back into teaching 
married women who have taught in the past, who have left teaching at one time or an- 
other and no longer have to be at home with their families during the day and who might 
therefore be available as teachers again. This constitutes perhaps the largest single 
resource that might be tapped in terms of the non -teachers today and the non -recent 
college graduates. But even this source, so some studies have tended to show, is Tke- 
ly to be less fruitful than many have thought; there seem to be not nearly as many such 
people that can be drawn back into teaching as one would assume . 

The Fund for the Advancement of Education has two experimental projects under 
way at the present time oh the use of teachers’ aides, sub -professional staff members 
who take over routine non -teaching functions and allow the teachers to devote a larger 
fraction of their time to the professional job of instruction that they have been trained 
to do. Few if any of the housekeeping chores are still placed upon the teachers. These 
chores are carried out by the teachers’ aides who can take the roll and monitor the 
study hall and do lots of other things adequately, but who can't do the teaching. Al- 
though this has a great deal of promise, I understand that there are a number of ques- 
tions in the minds of many in educational circles as to whether this type of program 
might not in the long run raise more problems than it would solve. But again, it is 
going to be within the next few years a question of what stratagems are possible and 
not "Is this the ideal way to solve our problems?" 

C--The nursing profession has used aides with considerable success. 
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THE PROGRAM FOR GIFTED CHILDREN IN 
THE PORTLAND OREGON PUBLIC SCHOOLS 

Robert C. Wilson 
Reed College 



— would like to describe to you the organization of the program and then some 
of the research studies we are carrying on in relation to it. The program was started 
in 1952 as the joint endeavor of the Portland Public Schools and Reed College. The of- 
ficial title of the program is ’’The Cooperative Program for Students of Exceptional 
Endowment. ” We call it ’’The Gifted Child Program, ” for short. 

A proposal was made to the Fund for the Advancement of Education to initiate a 
program for gifted children in the Portland Public Schools on a cooperative basis. The 
Fund looked favorably upon this proposal and made an initial grant for one year. Addi- 
tional grants ave been made to extend the experimental period to five years . We have 
now reached the end of the third year. I joined the project at the end of its first year. 

The project’s policy-making group consists of two assistant superintendents and 
two Reed College professors. Working with them are the Administrator Director, a 
former elementary school principal, and myself as research director. In addition, we 
have a small research staff and an elementary school consultant who visits regularly 
the elementary schools participating in the program. The program is presently oper- 
ating in fourteen elementary schools and five high schools, and this coming year will 
be extended to seven high schools which will include all of the comprehensive or neigh- 
borhood high schools in Portland. The largest proportion of Our funds goes to supply 
extra teacher time to be spent with the gifted. We add a teacher and a half to each 
high school, and a half teacher to each elementary school. The time afforded by this 
ia used to release regular teachers to teach special classes for the gifted, to develop 
activities for gifted children in the regular classroom, and to plan and coordinate the 
program in each school building. 

I’d like to talk a little bit about our identification procedure. We are working with 
roughly the upper ten percent in terms of intellectual ability in each of the high schools 
and elementary schools. In addition, v/e also include in our program the upper ten per- 
cent in each of seven special talents. These are art, creative dance, creative dramat- 
ics, creative writing, music, social leadership, and mechanical talent. 

One of the features of our identification procedure is that it does not operate with 
any fixed city -wide cutoff scores. Each school conducts a program for the upper ten 
percent of the children in that school. The absolute level of ability of the children par- 
ticipating varies from school to school depending upon the ability of the children in the 
particular school. Thus, a school with a student body of relatively low ability can have 
just as active a program as a school with a student body of relatively high ability. 

Identification goes on at all twelve gr ^ mploying whatever information is a- 
vailable. In the primary grades, we rely *a y on teacher judgment and have devel- 
oped procedures for systematizing their observations . Extra testing, specifically for 



the purpose of identifying the intellectually gifted, is centered at the fifth grade and the 
ninth grade. 



On the basis of scores from the California Test of Mental Maturity, scores from 
standardized reading and arithmetic achievement tests. teachers’ recommendations, 
approximately one -third of the fifth grade at each scl. x?l is selected for further testing. 
As a ceiling test, the SR A Primary Mental Abilities 1-*' -i 11-17) is administered 
under both time limit and work limit conditions to the upper third of the grade. Hie 
scores on all tests are compiled and children scoring consistently high, or those having 
exceptionally high scores on one or more tests, are included in the program. 



In the ninth grade we have been administering the Ohio State Psychological Exam- 
ination to the upper third of the grade as an aid in selecting the upper ten percent of the 
intellectually gifted. 



Next year we plan to experiment with the Iowa Tests of Educational Development 
as a means of identifying the gifted in high school. This test has the advantage of yield- 
ing scores in a variety of academic skills and may be more helpful in scheduling students 
into different kinds of special classes. 



Extra tests for identifying children with the seven special talents already men- 
tioned are given at various grades in the elementary schools. We have no special tests 
of scientific talent . Pupils are admitted to special classes in science on the basis of 
I.Q., interests, and teacher recommendations. 

The gifted child program has involved large numbers of people. I think part of 
its success is due to this. The program started out with a large workshop during the 
sumrr before the first year of operation, which was attended by teachers and admin - 

istrau and parents from the community. This group worked out the details of the 

program and the way in which it would operate. Two general provisions were estab- 
lished: 1) to encourage teachers in the regular classroom to do more work with students 
who have special ability or special talent; and 2) to develop part-time special classes 
for gifted pupils in both the elementary and the high schools. 

In the elementary schools we have special classes in foreign language, arithmetic, 
science, art, dramatics, creative dance, and a number of other subjects . These classes 
meet from two to five times a week, one period at a time. The classes offered vary from 
school to school depending upon the interests and abilities of the students in the particular 
school, and the availability of a teacher in the building to conduct a special class in the 
subject. The most popular special class we have is science. 

The most distinctive type of special classes in the high schools are the Junior - 
Senior Seminars. These are small classes of eight to fifteen students in the fields of 
mathematics, science, literature, and social science. The content of these courses is 
not prescribed, but is developed out of the student and teacher interests as the class 
moves along through the year . This means that the students get a chance to branch out 
into their own special interests . These seminars are characterized by emphasis on 
analytic exploration and interpretation of materials. There is very little of the question- 
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answer type approach or the lecture method. Members of the Reed faculty serve as 
consultants to the teachers and have assisted greatly in helping them to extend their 
knowledge in their subject fields . They also meet with the seminars periodically, and 
discuss informally with the students what they have learned. At Reed, the conference 
system is the most common system used in classes . This is applied here at the high 
school level in these seminars < 



A 



second type of special class in the high schools is the small section of a reg- 
ular subject. These classes may also employ the seminar method, but the freedom of 
content and exploration is not as great. At least a portion of the course must cover 
certain prescribed topics. Such classes are usually composed of 10 to 18 bright students. 



A third type of special class is the special section of a regular size class. An 
example of this would be an algebra class composed of thirty students all of whom have 
I.Q. ’s above 120 or superior mathematical ability. 



Slightly under a thousand students were enrolled in special classes in die five 
high schools that participated in the program last year . In the fourteen elementary 
schools our program affected slightly more than a thousand students who were enrolled 
in special classes. There were other students who had been identified for special talent 
or intellectual ability who were not participating in the special classes, but who received 
additional help from their regular classroom teachers. 



I think that this describes the general organization of our program. It is essen- 
tially an action program. The original philosophy was, "We won’t try to do a lot of con- 
trolled experimentation and then decide what to do. We’ll go ahead and do things and 
decide as we go along what things seem to work best. " 

We are doing research in three general areas . First,' we do studies which are 
aimed at improving our procedures for identifying gifted children, not only the intellec- 
tually gifted, but those with special talents as well. Second, we do studies which are 
rimed at evaluating differential educational practices used with the children participating 
in the program. Third, we do studies aimed at learning more about the characteristics 
of gifted children. Two or three of the studies we are doing or planning to do overlap 
the areas of this conference. 



This coming year we plan to do a study of the relationship between some of the 
tests of creative thinking, developed at U.S.C., some measures of conformity, and 
teachers’ judgments of creativity and conformity. I had hoped that I would be able to 
give you a detailed description of the experimental plan, but the details have not been 
worked out far enough so that I can do this. In general, what we will do is to use supe- 
rior high school seniors as subjects and have their teachers rate these students inde- 
pendently for their creativity and their conformity. Creativity will be defined as con- 
tributing original ideas, different points of view, and new ways of looking at problems. 
Conformity will be defined as doing what is expected and not disturbing or causing trouble 
for others. The students will take the tests of creative thinking and two independent 
measures of conformity. 
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A second study we are doing is a study of gifted underachievers. This is a fairly 
comprehensive study of about sixty pairs of boys and about thirty pairs of girls. . They 
all have X.Q. 's of 120 and above. The achievers (at least they are achievers as far £.s 
grades are concerned) have grade point averages of 1. 6 or higher, and the underachievers 
have grade point averages of 2.75 or lower. They average about 3.5. Our grading scale 
is 1, 2, 3, 4, and 5, with 1 as the top or equivalent to an A. These are freshmen and 
sophomores, who are being studied. 

We're studying many areas: the students' attitudes toward school, their liking for 
school, their liking of teachers, the characteristics they ascribe to the ideal, teacher, 
the school activities they take part in, what sort of study conditions they have to work 
in,, and so on. This latter will be through the use of a questionnaire. We're also using 
an adaptation of Brown and Holtzman's Survey of Study Habits. We are going to use ad- 
jective check lists in asking them to do an adjective check list description of their con- 
ception of the typical teacher, and perhaps one of their conception of the ideal teacher . 

In regard to their career plans, we are using a questionnaire on their future plans 
in regard to college, occupation, and their expectations in regard to their future careers. 
We're also using a modification of Strodtbeck’s Settle -for -Level -of-Occupation Scale. 

I learned of its existence from McClelland. It essentially consists of groups of occupa- 
tions going from professional down to unskilled. The subject indicates whether he would 
be pleased or disappointed if he were to wind up in each of these occupations. A best- 
fit line is drawn, the slope of which indicates the occupation level which the person would 
be willing to settle for. There seems to be some evidence that the level of occupation 
which an individual is willing to settle for is more closely related to his achievement 
than the level of occupation he says he would like to achieve. We're also using the V- 
scale or the Value -scale, developed by Strodtbeck and revised by DeCharm;, which has 
questions related to such values as: 1) delayed gratification of present needs and will- 
ingness to sacrifice some of one's present needs for a future career or for education, 

2) individual versus group work, 3) feelings about hard work and planning versus fate 
and destiny, and 4) willingness to leave one's parents. 

The third area we are investigating in this study is the area of the family. We 
have a questionnaire for the students on family relations. It concerns the treatment 
that students receive from their parents, parents' attitudes, and students' attitudes to- 
ward their parents . We also have a parent attitude questionnaire which both parents 
fill out concerning their attitudes and policies in regard to their child. 

The fourth area is the area of interests. We will use some of Dr. Guilford's 
factor -analyzed interest scales and also some self-descriptive dimensions of attitudes 
and personality. We also have available results from the Kuder Preference Test. 

In the area of social attitudes, we are planning to use some conformity scales, 
using parts of the E and F scales and a scale on the student's concept of the value of 
conforming. We are also using a gullibility or superstition scale. 

In the personal area we are giving a questionnaire on personal habits and activities 
such as health, sleep, eating, after -school jobs, outside activities like music, T.V. 



watching habits, reading habits, sports, hobbies, and attitudes toward religion. We 

also have the Mooney Problems Check List which is routinely administered to all high 
school students by the school district. 

We also plan to use some open-ended questions concerning goals and values, a 
self-conception adjective check list, and McClelland's adaptation of the T.A.T. for 
scoring of need achievement. We have also added twelve pictures which we are scor- 
ing in terms of the subject’s perceptions of social relationships. These pictures in- 
clude a mother and father with a boy and a girl, a teen-age boy and girl, a boy with 
an older man, a girl with an older woman, and so on. Various types of people are 
used so that the student can perceive various kinds of social relationships. We are 
doing a pilot study, using 25 pairs of high and low achievers, looking for differences 
between the two groups in an effort to develop scales that may be cross-validated 
with the rest of the sample. In addition, we hope to give some of the tests of crea- 
tive thinking factors we developed at the University of Southern California. 

About half of the instruments I have mentioned have been given. We didn't have 
time last year to give them all. Actually the amount of testing time is not more than 

about seven hours, and this is spread over a period of dme. Many of these instru- 
ments are quite short. 

Another study that I hope to do is a cooperative study with Dr. Guilford on a 
factor analysis of creative thinking tests administered at the seventh grade. We can 
administer some of the same tests to these fairly superior seventh graders that we 

originally administered to superior adults and factor analyze them to see if we net the 
same sort of structure. 

The last thing that I'll mention concerns the problem of early identification and 
encouragement. One reads everywhere that in order to increase our future supply of 
scientists we should identify potential scientists as early as possible and encourage 
them. By implication, we should identify potential creative scientists as early as pos- 
sible and encourage them. I think that in the present state of our knowledge we are not 
going to be able to identify with any degree of precision those who are most likely to 
become creative thinkers, or creative scientists. Nor are we likely to be able to in 
the very near future. In the meantime, I wonder if it might not be worthwhile to attempt 
to study and develop methods for increasing everyone’s ability to think creatively. 

I am going to conduct a seminar this fall with some elementary school teachers 
in which we will start out by discussing what is known about creative thinking. I hope 
that perhaps we can develop some methods and techniques that can be used in the class- 
room and taught to students. Is it possible, for example, that there is a creative 
method in the same sense that there is an experimental method? Are there ways of 
approaching problems which will maximize the likelihood of getting new and original 
ideas? And can these be taught in the classroom? 

If we succeed in formulating such methods next fall, we will try them out in the 
spring m several of our sixth grade classes. We will attempt to assess possible sains 
by before and after testing of the experimental group and a control group with adapta- 
tions of the creative thinking tests v,e developed at the University of Southern California. 



Q--What do they do in the science seminars? Do they have lab work — free lab 
work or something of that sort? Or are these just discussion seminars? 

S~No ; they do some lab work. They do reading and discussion and lab work on 
both individual projects and in groups. The teachers also try to give them some concep- 
tion of scientific method and the philosophy of science, which students don’t ordinarily 
get. I was quite surprised at the difference between college science courses and high 
school science courses. Generally, high school science courses seem to emphasize 
the technological aspects, experiments, and the results of science. There is often not 
too much emphasis on appreciation of the methods involved or the ways in which you 
draw conclusions. We have been able generally to get the best science teachers. In 
addition, we have had workshops every summer for the last four summers, in which 
hi#* 1 school teachers took courses in their own subjects. We have found from most of 
our teachers — and this is a criticism of teacher training institutions -- that they feel 
a great need for more background in their own subject matter fields. This is what they 
want. They don’t want courses in the philosophy and methods of teaching gifted children. 
If they are teaching math, they want a course in math. 

§ All of these courses are taught at Reed? 

S--Tlie workshops for high school teachers have been conducted at Reed, the 
others some at Reed and some not. We have in-service courses, too. 

Q - -Have you done anything to evaluate these seminars as to the amount of growth 
there might be in this ability to draw conclusions and appreciate the scientific method? 

S— This is one of the most vexing problems that we face. We haven’t been able 
to find any tests that seem suitable for this -- for measuring the kinds of things that 
the students receive from this sort of class. We have given questionnaires to the stu- 
dents, the teachers, and the parents, and we are doing some college follow-up studies. 
We have sent questionnaires to the students after they have had a year in college and 
asked them how they felt about the class and whether it helped them or not. Most of 
them felt that one of the values of the special classes was that they were better able to 
do independent work. They felt they had better research techniques and better study 
habits and that they were much better prepared in entering college than they would 
otherwise have been. Generally, students are over-directed in high school and under- 
directed in college. In a sense, we have made this bridging the gap easier. 

2 " -Do you have any control data of students that didn’t go through this program 
at all? Or did you plan any controls? 

S~Not yet, but we had a conference with some colleges last spring and we plan 
to do a comparative follow-up study of college performance. There are difficulties, as 
you can see, with the small numbers and the fact that many go to colleges outside of 
Oregon. These are the students who get scholarships, so they go all over the country. 

(J~Do you have any data at all on a control population? This is in terms of college 
performance and even in earlier interests and attitudes. I mean the ether 90 percent -- 
do you have any kind of selected sample of such students that you are following up? 
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S--We have one sample that we are following. It is the first fifth grade class after j 

the initiation of the program. Hie students will be entering the eighth grade this year. : 

We are following them along and we are just completing a study of their performance on 
the tests of arithmetic and reading achievement which they take every fall and spring. 

We have two groups that are matched for I.Q. and for their initial achievement in the 
fall of the fifth grade . We also matched them for the rate of gain they made during the 
fourth grade before the program was actually instituted. We haven't got the spring scores 
from last year yet to complete the study up to this point. 

Q — Along that same line you mentioned that there were some gifted children who 
were not participating in special classes. How many? 

S- -It's hard to tell. It’s probably relatively small. 

C--They are the ones that I’m interested in. 

S--I am and the people in the school system are also very interested in the under- 
achievers -- the children who do not get invited to participate in special classes. 

C — They are from poorer socio-economic status -- you can bet your bottom 
dollar on that! 

C — It ought to work the other way because you said there was a certain percentage 
by schools. Youngsters who live in the lower socio-economic group would, therefore, 
tend to get in faster because you take a percentage of that school if it works like that in 
our city. So the youngsters in the better side of town, relatively speaking, would be in 
the unfavorable position. 

S--Yes, in our high socio-economic schools the upper ten percent goes down to 
an I,Q. of maybe 128; at the lowest one of our pilot schools the upper ten percent may 
go down to about 105. Nevertheless, there are ten percent of the pupils at that lower 
socio-economic school doing something more than they would have done otherwise. 

Q — Are the techniques spreading to the rest of the teachers who are teachingother 
classes? Are they beginning to try to teach by the same methods your teachers are using 
in the special classes? 

S--Yes, some of these methods carry over. For example, the seminar teachers 
carry these methods over to their regular classes. Not only that, the program is spread- 
ing to other schools in the system that are not involved. This is something that makes 
it difficult for us. If a school decides that they want an in-service training program on 
the education of the gifted and it turns out that they are one of our control schools, how 
do we say "no" to them? 

Q--I'd be interested in the parental reaction particularly of the gifted youngsters 
who are brought in and also the student reaction. Are these children singled out now? 

Are they in any way categorized? 
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S-~We were very concerned about this — the community’s reaction and.whether 
children would be singled out as ’’brains” or something like this. . We have had very 
good acceptance of the program in the community. In questionnaires we gave to the 
students, we asked them if participation in a special class made any difference in their 
relations with other students, i. e. , whether they felt they had gained in prestige, it had 
made no difference, or it had hindered their relations with other pupils.. Overwhelmingly 
they said it had made no difference. They do not perceive themselves as being set apart 
and many of them said, ’’Why should it?”. One thing which helps is that there is a large 
enough percentage of students involved that they are not singled out too much. In the 
elementary school this was so inconceivable to the pupils that they couldn't understand 
what we meant when we asked them if it had made any difference in their relations with 
other pupils. .. . . , 

Q-.-Are all. of the classes in the elementary school separated out or just some of 
them? ... 



S The pupils spend most of their time in the regular homeroom class. They 
leave the homeroom class two to five times a week to take a special class for one period. 

Q hi the high schools, are most of their classes in the regular program along 
with the other students? 

S- -That's right. They generally engage in only one or two special classes. 

2" -One question I would have is when you begin to define these special abilities 
in art, music, etc., do you note you begin to cut across almost the entire population? 

In other words, if you are going to take the upper ten percent on the California Test 
of Mental Maturity, then you get a restricted population, but when you begin to define 
special talents in terms of primary mental abilities and the like, don’t you almost get 
everybody in the upper ten percent on something? 

S--Well, we haven’t actually been identifying the upper ten percent on separate 
primary mental abilities . We have, however, been trying to pick those children who 
show talent in the various talent areas. These cut across the range of intellectual 
ability. In other words, a student who is gifted in art or dramatics may not be partic- 
ularly bright in terms of intellectual brightness or I.Q. 

C--So the ones that are taking the dancing and the art may not H in the sample 
that you have chosen in the intelligence tests. 

S--This means that we actually affect more than ten percent of the total school 
population. 

Q--Have you measured the amount of overlap in these various areas? 

S— No, we haven’t and I suppose it might be worthwhile. I, at one time, thought 
we could shed some light on the controversy about the relationship of I.Q. to special 
talents of other kinds . 



C- -There is no doubt that the intellectually gifted are gifted in many other ways 
but that there are many others that are gifted that do not fall into this particular class . 

C--Of course, you also end up with another kind of confounding variable: that is, 
the high intellectual student -- high in verbal facility -- can also do a series of other 
things to increase his visibility in a classroom which can compensate for low socio- 
economics or other factors. Thus, we can set him up in front of the group and here he 
gets a lot of encouragement for that, whereas someone who may be talented in other 
expressive factors such as in dramatics and dance, which may not be correlated with 
verbal facility, does not have that specific talent encouraged. 

S' “Well, we are also trying to encourage these other talents. We are, first of 
all, trying to give all students a chance to demonstrate whether they have any talent 
in these areas. We give our talent tests or screening exercises to all students. Then 
there are special classes and other activities for those who show up well. 



SOME MEASURES RELATED TO SUCCESS’ AND PLACEMENT 
IN BASIC ENGINEERING RESEARCH AND DEVELOPMENT 

David R. Saunders 
Educational Testing Service 



S- -Although the work that I want to talk about here is more than somewhat re- 
lated to the general problem of creativity, the actual purpose of the study on which 
I'll base my remarks was somewhat different. The purpose of the study was to develop 
information that would assist in the best possible placement and utilization of engineers 
as a group, rather than of creative talent as a pool of creative talent, and for that 
reason a lot of the data we have collected has to do with engineers who are engaged 
in selling, supervising, manufacturing and other function, as well as the basic research 
and development which is referred to in the title that has been given you. There is un- 
doubtedly a lot of information here that bears on the question of creativity which I’m 
going to try to pull out from the other material that we have . 

The study has been carried out cooperatively with the sponsorship of a group of 
five companies who employ a large number of engineers . I think it would be appropriate 
for me at least to mention these companies here. They are the Beil System, Detroit 
Edison Company, B. F. Goodrich Company, IBM, and Westinghouse, and we owe them 
a debt of gratitude for their far-sightedness, I think, in undertaking to support this pro- 
ject and to provide the subjects and other things that are necessary to its being carried 
out. 



The general approach that we've been taking is this: We've started out with en- 
gineers who have five to seven years of experience beyond the time when they graduated 
with bachelor's degrees in engineering, and our first major step was to find out how 
these experienced engineers may be differentiated. We have two main dimensions to the 
criterion along which we're trying to differentiate. We have the dimension of the actual 
kind of work they are in -- i.e. , their functional job placement and we also have the 
dimension of their success in performing this work. I'll be talking some about both the 
functional aspect and the success aspect of the criterion. 

By way of background to my later remarks, I'll say something first about our work 
on the criterion side, so that we know what we’re shooting at. We had to develop some 
methods for defining our job functions, for identifying individuals who were working in 
them, and for assessing their success in doing this. After these have been discussed, 
there will be two -- perhaps more -- rewarding aspects to my remarks: one in terms 
of what correlates with placement in the job functions that are most closely related to 
the subject of this conference, which appear to require some creativity, and secondly, 
what happened in the Study to a group of three tests which could be regarded in some 
sense as tests of creativity. So let's get on to the criterion part of the problem. 

We worked up a device which we call the Key Group Data. Form as a means of ob- 
taining objective job definitions . This form consists of a list of twenty-four activities 
which almost any engineer might engage in. For example, there are such things as 
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"completing experimental or pilot project, " "conducting negotiations, " "controlling ex- 
penses, " and "developing and testing useful hypotheses or generalizations. " These 
would be four of the activities. We also included some social activities like "sizing - 
up people, " "selling ideas to people, " and "participating in technical society or com- 
munity activities. " Another activity that we'll be concerned with is "originating tech- 
nical ideas." We asked various people -- individual engineers, their supervisors, and 
people in personnel departments --to rank -order the entire list of activities in relation 
to a particular job, i.e. to indicate how crucial it is for an engineer in the particular 
job to perform these activities by ranking them from top to bottom in importance. Such 
a ranking constituted somebody's description of some job. We were able to get into 
quite a detailed analysis of the relationships between different person's judgments about 
the jobs and by various procedures we were able to arrive at a composite description 
of each of the twenty-three jobs in the five companies with whom we were participating. 

-Who contributed the itemd in the list of activities? 

S--The list was based on analysis of a number of more conventional job descrip- 
tions. We tried to include all the variety of elements that are normally taken account 
of, and then we tried to generalize these -- reworking them and getting them into a 
form that would be psychologically amenable to interpretation. 

C--The thing that crossed my mind is the very careful observational studies, 
like those in the Bank Wiring Room of Western Electric, as to what these people actu- 
ally did versus their reports on what they did. I don’t know if you have any way of 
getting into the situation. The social perception of what goes on is not always accurate. 

S- -That’s true, and we didn’t get into the situation to see what they actually did. 
However the purpose we had in mind was simply to provide a basis for sorting jobs 
out, and we might expect that if the job was the same, even the social perception of it 
would be the same. Whether this was, in fact, an accurate description was not crucial 
for us. We hoped it would be, and at one point I'll mention how we attempted to capi- 
talize on the fact that it might be. We collected these data; we analyzed them; we had 
a composite description of each of a number of jobs. Our purpose was simply to sort 
out the jobs and try to group them to show how certain jobs in different companies were 
essentially similar whereas jobs in the same company were different, and to have an 
objective means of doing this. 

The part of the result that is of concern to us here, I think, relates primarily to 
the first two of the seven categories of jobs we came out with. The first category in- 
cluded five different jobs, in four of the five companies. The jobs were given titles 
by the companies such as "research, ’’ "development, " and "product development. " 
Characteristic of these jobs was the fact that, first of all, originating technical ideas 
was highly characteristic of this group, and relatively less so of other groups. Secondly, 
the development and testing of hypotheses and generalizations was important. Then in 
the next category, we had four other jobs out of the twenty-three, to which companies 
gave such names as "development, " "product improvement, " "design, ’’ and "manufac- 
turing. " (These become a little diffuse, and you see why we had to get into a method 
like this to try to identify which jobs would match up across companies. ) Here, there 
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is only some emphasis on the ’’origination of technical ideas, ’’ and no emphasis on the 
"development of usefuJ hypotheses and generalizations.” However, throughout both of 
these functional groups there is a lot of emphasis on being informed about various tech- 
nical matters and being able to think clearly. There is relatively little emphasis on 
social skills, and little emphasis on such things as controlling expenses or developing 
personnel toward promotion -- these things are associated with other types of jobs. 

, , Here, then, was a beginning. We were able to use this in two ways. One way 
was to help us in sorting out jobs and assigning data, that we would collect later, to the 
proper functional category. Secondly, on the implicit assumption that the descriptions 
were valid, this provided one basis for deciding what tests we ought to develop that might 
help us to separate these functions. In this regard, the activity of "originating techni- 
cal ideas" appeared as a particularly important one in our data. Although we did not 
do this, one can conceive of considering the activities of our list and doing some sort 
of factor analysis to see how they group themselves. If we were to do this, I'm pretty 
sure that we would come out with "originating technical ideas" right at the center of 
the biggest factor we would have. This one activity runs right straight down the total 
list of twenty-three jobs; you can almost order the whole set of jobs in terms of how im- 
portant it is to be "originating technical ideas . " The "development of hypotheses and 
generalizations" seems to operate the same way, but it cuts at a much higher level as 
far as our data are concerned, and actually it’s within the most important three activ- 
ities only for groups that were literally labeled as research by the companies. 

Q--How do you differentiate between the development of hypotheses and the orig- 
ination of ideas? 

S--I don't differentiate. They are the phrases that were used in the form, and 
they mean whatever they meant to the people who filled out the forms -- there were no 
definitions provided for these things. It was going to be a big job to fill these out, at 
best. 



Q--Were there definitions for the people who decided whether a given activity 
was in the job description or not? 

. S--The notion was simply that these phrases had to be self-explanatory, and we 
got very little kick -back when we used,.them. 

One indication of these results is that if we want to differentiate the researc’ nd 
development functions, and perhaps also if we want to predict success in these functions, 
we do need measures of creativity. I have gone into this because it represents an ob- 
jective bit of evidence for this thing that we have known anyway. This represents about 
all I plan to say about the criterion development aspect of this study. 

Q- -Somewhere along in there, I presume you visited these companies. In your 
deliberations, did any issues crop up about ways and means of assessing the creative 
components of a particular design or an idea or a piece of machinery? 

S- -There was a lot of difference between these five companies as to how much 
emphasis was placed on the problem of creativity in their own thinking. One of the 
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companies was interested almost solely in creativity -- and in differentiating between 
these first two functional groups that I have emphasized here. Actually, we didn't have 
anyone to test within that company except members of these two groups. 

2" "What 1 mean is did you ever crack this thing down to where you are confronted 
with a piece of machinery or innovation or a design? 

S.~ ~No • Another aspect of this thing was that we needed a success criterion that 
could be applied equally well to all of our functions. We couldn’t emphasize too specific 
an aspect of this criterion, especially if it was likely to get into the criterion for only some 
of the functions. To this end, we said to the supervisor who was going to rate the man, 
"Is he the kind of person that you'd like another person to be like" Is he generally suc- 
cessful compared to his job requirements?” And then we had to assume that he was being 
evaluated largely with reference to the requirements of the job he was in, for which we 
already had a description. 

C - -I m terribly worried about this . A formal statement of a job, and the actual 
behavior that goes into the job, can be two very different things. The kind of informa- 
tion you get may depend only on the particular bias of the particular person you happen 
to ask. 

S--A sidelight of our study that would certainly support your point would be that 
in one of our other devices we asked various people to rank different personal qualities 
according to how important they were for these jobs, and one of the qualities was "re- 
sistance to boredom. " When people in the personnel division would rank this vis-a-vis 
the function of being a research engineer, it would rank very high. When an actual re- 
search engineer or his, supervisor would rank it, it would rank very low. 

C--That's exactly the kind of thing I mean. That's just what comes out of Harvard 
case studies, and it would lead to some terrible maladjustments, that is, a guy would be 
told by the hiring people, "Now, we really want you to be creative and we hired you to do 
that." And then he gets in the job and finds that he can’t be, really, because the way 
they’ve got it set up they don’t really expect him to be . In fact, at any particular time 

they expect him to do something else, and yet all the time there is the big pressure for 
creativity. 

S--I think this is part of what we felt when we resisted as much as we could any 
pressure to put up a criterion that said we were measuring, for example, creativity. 

We simply were trying to measure whatever it is that is really wanted in those jobs. 

C- -Sensitivity to boredom would probably be a very worthy thing. 

C--You see, your criterion very precisely stated that you are now going to try to 
predict what people think is important in this job, which has an indeterminate relation- 
ship to what is important in the job. I assume there is some correlation, but you see 
the problem. 

2“ "Isn’t there a question of how far from the front office he has gone for his data? 

A ^ 10 are t ^ le supervisors? Are they men very close to the work, or are they men very 
far away from it? 
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S-The supervisors we are concerned with here are the first line supervisors of 
the actual engineers whom we were going to test. We also collected from these latter 
engineers their own impressions of the importance of these activities, and correlated 
these with the composite picture we had from the supervisors. For the most part, 
there was a significant positive correlation, but perhaps it wasn’t as large as onewould 
have liked. There were a few engineers, of course, whose own descriptions would cor- 
relate -.40, or something like that, with what the supervisors as a group thought this 
job would require. Nevertheless, it is true that if a supervisor claims a particular 
activity is important, and ranks it up among the first three, this also tends to be true 
of the men, the actual engineers. There is a fairly high correlation between the rank- 
ings of all the men on their own jobs, and their supervisors’ rankings. In the later an- 
alyses, we did tend to eliminate the individuals whose own description disagreed really 
violently with the description of the supervisors. 

2" -Do you have any measure relating the discrepancy between a man’s own eval- 
uation and his superior’s evaluation of his job to success on the job? 

S— This is something we want to look at sometime. We haven’t yet. 

Q — Does this bring us back to the earlier statement that if you have a discrepancy 
between the formal statement and che informal behavior, it can be a measure of internal 
conflict and also of success, role and role -fitting sort of thing? 

C— If you went to most graduate schools and asked the members of the staff whether 
originality and creativity were important in' their evaluation of their Fh. D. students, Fm 
sure they would all say, ”Yes.” And if you asked the students, they would agree that it 
is important. And yet, I don t believe either one of them for a minute. 

C--I don’t believe that the students would agree. 

C — I’ll bet they would. 

C~I know one graduate Dean who once said that it i^n’t important; only endurance 
is important. 

C-He might say that privately to you. Nevertheless, if you went to him in an of- 
ficial capacity and put him on the professional spot, he’d say, ”Yes, ” for his graduate 
school. 

C— But over a beer, he’s a devotee of endurance. 

o--Here s a statement in support of these remarks. We asked a number of math- 
ematics professors at a number of universities to rank -order sixteen foctors for learn- 
ing graduate mathematics for Ph. D’s. Originality came out next to the top among the 
sixteen, and yet it was found that the correlations with the criterion were zero. 

S- -Actually, there are two kinds of information on this Key Group Data Form. 

I didn’t mention the other one. We asked the supervisors and engineers not only how 



crucial each activity was, but how much time they actually spent on it. With respect 
to such things as originality and hypothesis testing, etc. , there was probably quite a 
discrepancy between the proportion of time and the importance attached to it. 

C~We have a small bit of evidence on thi3 also. In studying Air Force officers, 
we used a list of some fifty traits which we asked superior officers -- officers of the 
general level of Colonel or above --to Q* sort in terms of what they would look for in 
really good officers. And we also had officers of the rank of Captain, whom we were 
studying, Q-sort the same things in terms of what was required of them. There was 
a big difference in where they placed originality. "The Captains generally placed it well 
below the mean as an important trait in their work, whereas the superior officers thought, 
or said they thought, it was a very important variable. 

C--I've got some hard, bitter experience to bear that point out. I once pulled a 
two -week training tour, and the Colonel called me in his office, and disregarded the 
fact that I was in uniform. As far as he was concerned, I was a civilian in research. 

He said, ’’What's the matter with my people? I’ve got to dream up all the ideas. I 
can’t get any ideas out of them, any of the originality that we need for reorganization 
of the basic training program. " I think it bears out that discipline was of a high order. 

S- -Shall we look and see what happened next? The next thing that we did in this 
study was to develop a battery of measures -- we had some fifty-six in all -- which were 
then administered, scored, and analyzed with the idea of finding out what would discrim- 
inate between the functions, and what would predict the success ratings that we made 
within the functions. We had quite a lot of interesting results, most of which I'll skip 
over. 



In general, we found that Function I, the research or basic development function, 
,was the easiest to work with, in terms of separating it from the others and predicting 
success within it. The next easiest was sales- The third easiest was the development 
function which is Function II, the other that I mentioned earlier. Supervision was also 
among the easiest. What are the things that may be used to discriminate the more or- 
iginal or creative engineers, if they are this, from other engineers? First, I can men- 
tion a test that was introduced into the battery specifically as a measure of inductive 
reasoning, and which is a glorified number series affair. It presents items of this 
general type: You are given sets of matched values of X and Y and perhaps Z — several 
sets -- and then an incomplete set. You are required to supply the numerical value of 
the missing variable to complete the last set. You can do this either by just following 
it through as one tends to work number series, or you can fit an equation to the sets of 
values that are given to you and solve this equation to obtain the missing value where it 
is needed. This particular test is very powerful for picking out the research people in 
contrast even to the development people -- the development people are just like all the 
other engineers on this, whereas the research and fundamental development people stand 
almost a full standard deviation higher with this particular measure. 

This may be seen in Figure 1. The pairs of vertical boxes represent high and low 
subgroups within each of the functions. The left-hand pair of boxes that are crossed by 
a heavy horizontal line respectively represent the low and high parts of the research 
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function. Then, the next pair represent the low and high parts of the development func- 
tion, labeled II. The other boxes, or bars, represent other functions, miscellaneous 
technical (IV), operations (V), supervision (VI), and sales (VII) as you go on across. 
(The tops and bottoms of the bars are plotted at plus and minus one standard deviation.) 
You can see that Function I, research, is far ahead of the others. Even the poor people 
in that group seem to be doing better on our measure of inductive reasoning than people 
in other groups . (The two small negative correlations are insignificant . ) You will note 
that it is still a fairly brief and rather speeded measure we have been using in our re- 



search, and that it requires some lengthening before it can be applied to the measure- 
ment of an individual. As a by-product of that lengthening, there would probably be 
some increase in the apparent separation of the groups. 



The effect may very well be due to practice. We're working with engineers who 
have five to seven years of experience, and they are in these functions right now. Even 
within the function, lowever, the successful people are much higher than the unsuccess- 
ful people. 



Altogether, there are about 800 individuals represented in the figure. There are 
100 individuals represented in the high part of the research function and fifty individ- 
uals represented in the low part of that function. Similarly, this is true for the other 
functions, except for IV which was not divided into high and low because there were 
only fifty cases in it altogether. 

Success, as you will remember, was simply a supervisor’s judgment as to 
whether this engineer is the kind of person he likes to have in his group. The ratings 
were initially on an eight-point scale, but were arbitrarily divided into a highest two- 
thirds and a lowest one-third for these analyses. The split was made this way as a 
sort of insurance in case the truly poorer people in the group were few in number. 

There probably is some difference in formal training between engineers in 
Functions I and II, for example. I don’t know how much. We controlled on the length 
of experience since the bachelor’s degree, and we considered that a master's degree 
or doctoral training was part of that experience. There are undoubtedly some effects 
from that source, but I don’t think they are too large because a relatively small pro- 
portion of our engineering people actually go on for any kind of advanced training. 

S’" D° Y° u Aink if you had a nonconformist v/ho was needling him all the time to 
do something else rather than what the department was set up to do, that the super- 
visor would say this is the kind of man he would like to have another man like? 

S--Well, this may be, I should think it would vary with the supervisor. 

Q--Did the supervisors take these tests? 

<C--No, they never do! 

2— I’m interested in the criterion of success. Was this a matter of personal 
compatibility? Did that enter into it or was it purely a matter of a man’s competence? 



S**~It should have been purely a judgment of a man's competence, but I have no 
way of knowing that it was. There were probably a lot of things that got into it, and we 
may see what some of them are as we look at some of the other variables that come out 
significantly. This measure of inductive reasoning, which is largely an intellectual 
quality, is better at separating functions and predicting success within the research func- 
tion than the more conventional intellectual measures we had included, such as verbal 
analogies as a measure of the verbal and reasoning factors — such as a numerical 
test solving problems of the conventional kind, for example, if two men can build a fence 
in two days and three men can build it in a day and a half, etc. 

Did you use a mathematical analysis test? 

S--We didn't have that. We did have to pick and choose somewhat for the tests 
that we included. 

2'"°° y° u have idea why the number series turned out better than the problem- 
solving one? 



S— Well, I would like to offer an hypothesis or two here. I think that it was a little 
more difficult, although the ordinary computational problems test was difficult enough 
by conventional criteria. Even in die latter, the engineers didn't get all of the items 
right in the time allowed; it did seem to be a power test for most of the functions, and 
y®^ it was a speed test for the research and development people. Perhaps we didn't 
give them enough time to generate a big difference on that measure. 

2" "I can suggest another reason. I don't know what your number series test was 
like, but in our experience a number series test is quite complex factorially. 

^ That s true, too. I have the test here if anyone wants to look at it. 

2” Was the difference partly a difference between calculation, which is simply a 
procedure of stepping from one point to another as fast as possible, and the perception 
rather synoptically of pattern, that is, the better test involved a perception of pattern 
rather than the use of a series of skills or insights? 

S~ That may also be true. You are certainly required to form some kind of gen- 
eralization, either explicitly or implicitly, to arrive at the answer in this number series 
test. 

C— By the way, Science Fiction publishes some problems called "Nimbles, " or 
something like that, which are something like this. 

S--A second major group of tests that turned up big differences between our func- 
tions, and contributed to the ease of separating the research and development groups 
from the other groups, were essentially measures of interest. Here, we started out 
with an hypothesis as to certain interests that it would be important to measure. These 
were interests in ideas, in things, in people, and in economic affairs. These were ex- 
pected to be important for discriminating our whole spectrum of job functions. Further- 
more, we had specific hypotheses as to the relative order of these interests for the dif- 
ferent groups. For example, for the research group, we expected that interest in ideas 
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would be at the top and interest in economic matters would be at the bottom. For the 
sales engineers we predicted just the opposite, and for the groups in between we pre- 
dicted certain rank orders that turned out to be pretty well -represented in our data. 

Let's look at what happened to the measure of interest in ideas. (Figure 2. ) The 
research and development group is way out in front with a substantial positive net score; 
there is also a very high correlation with the success ratings within the function. 

This test was constructed by putting together items that had five alternatives . 

Each set of five alternatives contained one relating to ideas, one relating to things, one 
for people, one for economics, and one that was supposed to be unrelated to any of these. 
For example, one of the items, actually the first one in the test, and the one that turned 
out in the item analysis to be just about the best one in the test, was: "If you were to 
visit a textile company, which one of the following activities would interest you the most? 
(A) Methods used to test the strength of materials. (B) Methods used to predict the prop- 
erties of the new synthetic fibers. (C) Methods used to increase public demand for syn- 
thetic textiles. (D) Methods used to handle public relations. (E) Methods used to trans- 
port raw materials. " Now, the key probably is that testing strength of materials would 
be an interest in things; predicting properties would be ideas; increase in public demand 
would be economic; handling public relations would be people; and transporting raw ma- 
terials is not supposed to be of too much interest to these engineers. 

For each set of five you were asked to select the one that would interest you the 
most, and the one that would interest you the least. You were given plus 1 for ideas if 
you selected an idea alternative as most, and minus 1 if you were to select that alter- 
native as least, so that you would expect a score of about zero on each of these interests. 
When I spoke a minute ago of a positive net score, it simply meant more "mosts" than 
"leasts . " There are fifty sets of alternatives in the test, making a hundred responses 
to be made. 

This test was completely a priori in its keying, that is, we wrote the items and 
keyed them immediately on the basis of judgment. The results in this figure are based 
on scores that came from those keys. Item analysis has been carried out since, and 
the results that can now be gotten from the test ought to be considerably more striking. 

Q--The item you just read seems to be very close to the content of the actual work. 
Are there some that are directed at a different level? 

S--I think there are, although I'm not sure whether these are the items that fell on 
their face in the item analyses or whether these are other items that do some good. Here 
is one: "Which of the following would be the most important consideration before trans- 
ferring somewhere else to anew job? Increased salary? Leadership responsibility? 

Ease of getting to work? More up-to-date facilities for work? Or interest in work?" 
(Offhand, I'm not sure how this one was keyed. ) The item I read first obviously contains 
material related to what the men are doing in their jobs, but it is not identical content 
because the companies that were participating here were largely electrical companies, 
not textile companies . Right now our subjects need not be concerned with such things 
as strength of synthetic fibers or other things directly going on in a textile company; 



however, if they were to get into a textile company these might indeed be the things 
most closely related to their work. It’s perfectly true that you can see through this 
thing, and it definitely needs to be longitudinally validated. 

< 

At this point, the indications are that these interests do relate to and match the 
various functions that we have. There is probably not too much direct relevance to the 
problems that we are talking about here. Let’s go on. 

Along with a number of tests that we included on the basis of more or less spe- 
cific hypotheses, we had a "shot-gun" test of various personality variables which might 
be related to the criteria, but about which we didn’t have any specific hypotheses. Well, 
Figure 3 is for "talkativeness"; the relationship is in the direction of salesmen being very 
high and good salesmen being highest of all. There is also perhaps a little tendency for 
the people in the research and development functions to be rated higher when they are 
more talkative and can express their ideas. This is a personality inventory measure. 

All of these next measures are inventory measures, and are the result of summing the 
responses to ten items such as "Are you a relatively talkative person?" "Do you talk as 
readily in a group often as in a group of two or three?" etc. 

Q--Or "Do you tell the truth when asked if you are talkative?" 

S--We didn’t have that one. We recognize that these are purely self-ratings. The 
instrument was administered in the research situation and, for the grouped data, I think 
our results are somewhat trustworthy. The difficulties in using inventories in selection 
situations, because of the way some people respond, need not be feared here. 

Now let’s consider the next one. This is one we might do some predicting on. 

This is going to be a measure of liking to think, so what sort of relationship should we 
expect? These items are perhaps a little bit more subtle than those for talkativeness. 
Here is a representative one for you to evaluate: "Are you inclined to analyze the mo- 
tives of others?" There are nine other items in the scale along with this one, which 
happens to be statistically the most representative one. The "liking to think" desig- 
nation is an abstraction which is supposed to fit all ten. 

C--I think that item would be a good one for research psychologists. 

S--Here are two more items: "Do you enjoy reading and thinking about such topics 
as what are the most valuable things in life, what is life all about, what are people here 
on earth for?" "Which would you rather be, a politician or a lawyer?" These and the 
other items all inter cor relate positively; the scale is factorially developed. 

C--Then this should not show any difference between your research and the other 
groups . 

S--The key on this last one is that you should want to be a lawyer, if you like to 

think. 



C- -Certain aspects of law are research in themselves. 
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C- -But if the key to analyzing other peoples' motives is liking to think, you would 
think politicians would carry a high rating. But politicians' thinking is not creative. 

C- -Except if you run the Allport-Vernon against high-lows in industry, when you 
find with a latent structure analysis that the political item does get in, because of the 
need for dominance. To be effective creatively in an industrial research organization, 
we need the political side. 

C--That effectiveness is not really a creative effectiveness, but a dominance ef- 
fectiveness. 

C--Give us the results! (laughter) 

S--The differences in liking to think (Figure 4) are not huge, but the research 
group is the highest and the sales group is the lowest. The supervisor group is the 
next highest here . The good research subgroup seems to be a little better than the 
poor research subgroup on this. 

No one of these is very important, so we'd better pass over them fairly rapidly. 
Figure 5 shown next is labeled "Tolerance of Ambiguity. " The most representative 
item is this one, "Is it hard for you to sympathize with a person who is always doubting 
and unsure about things?" As you can see, research people find it easy to sympathize. 

1 C--Y 0 U know, it seems to me that a salesman should be a master of tolerating 
ambiguity — if the customer doesn't want to buy, he keeps selling just the same. 

S--This is what we find. It is as if neither the researcher nor the salesman know 
what’s going to happen as an immediate result of their specific actions, and they put 
up with it. This scale has no correlation with success at all. It's related purely to 
function; in fact, the F-ratio for success within function was practically zero for this 
one. 



Q- -Do you hjve any idea if you took a professional breakdown for type, like mechan- 
ical against electrical engineering, etc. , whether these things split differently? 

S--I’d have no idea on that at all. Our breakdown by function cuts right across 
any breakdown by field. For example, Roman numeral I or n can include mechanical 
as well as electrical, etc., as a mixture. 

Let's look at Figure 6. This one is for masculinity versus femininity. In our 
data, this is correlated with another scale which is practical versus artistic. High 
scores are for masculinity, and should be interpreted as indicating an interest in vig- 
orous rather than passive pursuits. The label of masculinity has been given to this 
scale because it correlates with sex about .7, which is much higher than any of our 
other scales. A sample item is: "If you could do either equally well, how would you 
rather work? Guiding young people seeking careers, or managing a manufacturing 
concern? " Another is: "If you could be equally successful as either, which would you 
rather be? A parachute jumper or an actor?" 
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This scale has a significant interaction effect between the success rating and 
function, which implies that there is correlation with success in some of the functions 
and that this correlation varies from one function to another. Within research, we 
seem to have a negative correlation. Good researchers prefer relatively passive 
activities and are more artistic. 



n--Tg oniiHinor unnnor npnnjp cnnnncPfl tr\ Hp fpmininp? Tt c /=»/=» rnc tr\ mg that* that 

item has an altruism dimension running through it, more so than the masculine - 
feminine. 



S--It may, although we have another scale here which we call altruism, and it 
is not correlated with that. 

Q--This was an a priori key. Did you actually give this to women, or were the 
items taken from a standard M-F scale? 

S-’The items were not taken particularly from any sources. But the items do 
correlate with what sex you are. It has be§n given to women, and the women do score 
distinctly on the feminine or passive side. The mean score for engineers here, you 
notice, is twelve or thirteen. If you go to a mixed group of men and women, the mean 
score will only run around ten. 

# 

C--I hope you make something of this because this is quite an important finding. 

You see, a standard finding ever since the Strong first came out is that engineers score 
among the highest in the masculinity-feminity scales. If you start using that as a screen- 
ing device you might screen out your most creative engineers who are a little atypical . 
This is one of the things that I want to talk about. Whereas the criterion holds for most 
of the range, it may actually reverse itself at the very top. 

S--Well, within our Functions I and II, there certainly seems to be a negative 
correlation of masculinity and the rating. Within the other functions, there seems per- 
haps to be a positive correlation. The creative engineers are mainly masculine, based 
on the general population scale anyway, but they aren’t extreme as compared to engi- 
neers. 

Figure 7 we call ’’self-acceptance. ” Perhaps the low scores represent self-crit- 
icalness. Again the research group is toward an extreme. The researcher seems to 
be a little more critical, a little less self- accepting. 

C- -Incidentally, this is very nice minor support for that article by William Whyte 
in Fortune magazine. He probably didn’t realize it, but this is exactly what he said. 
Sometimes your ordinary people will score higher in self-acceptance than your very 
good people. Of course, he had no evidence, except subjective impressions from talk- 
ing to people at the top that they tended to be more self -critical than the ordinary run- 
of-the-mill people. He was using this to attack psychologists who use inventories. 

C— You seem to have a slight negative correlation here again in your first two 
categories . 



S--No, the analysis of variance says that this scale contributes purely to func- 
tional discrimination. 

I promised to say what happened to some creativity tests we included. It does 
seem to be relevant to what we are talking about here, but there is time to' cover this 
only briefly. 

We included three tests that might be called creativity tests. I haven't mentioned 
any of them yet. One of them is essentially a consequences test. One of them is a con- 
trolled associations test. Both of these you are probably familiar with from the reprints 
that were distributed. The third one is called an "Idea Classification Test, " and works 
like this. You present the subject with a list of words, which stand for things, of course, 
and the task is to find subgroups of these words. You are presented with twenty -five and 
you are to find subgroups of four which belong together for some reason, and you state 
the reason. You are urged to find as many such subgroups as you can, but not to form 
them on the purely mechanical basis of word-structure, etc. We had relatively poor 
success in using these measures to predict the things we were interested in. That's why 
I haven't mentioned them already. However, there were some things that they seem to 
be good for . 

In all three of these tests we followed up a notion that seems to be implicit in 
Guilford s originality factor, that the difference between an obvious and a remote re- 
sponse, between a frequent and an uncommon response, between a banal and a clever 
response, was important. We distinguished these. We are glad we did or we would not 
have had anything from these tests. In the Controlled Association Test, the remote, 
uncommon response^ gave us nothing; whereas, the number of obvious, common re- 
sponses, turned out to be the best single predictor of success, independent of function, 
in the whole study. Perhaps this can be explained in terms of an ability to communicate 
effectively by relying on the more common associations of words. The more responses 
you have given which are among the 50% most common responses, the more successful 
you will be judged. This finding may also have something to do with supervisors and 
what they are judging. It may not be a predictor that you really want to use. 

Q- -I am thinking about the test where you are given a stimulus word, "bad, " 
and you have to think of as many other words that stand for the same thing as you can. 

Is this the same test? 

S- -That's right. 

3 - -Well, that's interesting. In another study wherein we used a completely dif- 
ferent criterion, we found that associational fluency, controlled association tests, gave 
us the best prediction. Our criterion was instructor performance, where a man gets on 
his feet and instructs a class. This requires the associational fluency. 

S--This test is not pure fluency. Fluency would be best measured by our total 
score counting all responses, and that does not work. 

C--We got our best results out of the total score. 



S--Our operational definition of a common response was this: We rank “ordered 
all the responses to each stimulus word according to frequency. Then we found the 
number of distinct responses that we had to use which would bring in 50% of the total 
frequency of responses. These responses that would account for 50% of the responses 
were called obvious. " These common responses are keyed as ’’correct. ” 

There probably is fluency variance in that score, but there would also be fluency 
variance in the other half of the score. 

C- -But more originality. 

S- -Perhaps plus originality. But the originality that you come out with, it seems 
to me at this point, is meaningless originality. It hasn't got a quality that it needs to 
have to be good originality. 

Q~ ‘You probably have the data to do this, but did you get a discrepancy measure 
between the commonness of the response and the originality? 

S““No, the two part scores behave quite differently. The commonness score was 
significant in differentiating the successful from the unsuccessful in all six functions 
that we have data for. The other part score was insignificant in all instances. The 
criterion still was how well you would like to have this man on your staff. 

In this test you have a restricted situation. There are a limited number of stim- 
ulus words that are actually used. Synonyms for the original words are' probably here 
in the commonness score; the more original responses may show that a person doesn't 
have as good verbal comprehension. 

Q~ -Incidentally, how long does that test last? 

S- -As we gave it, it was a twelve -minute test. 

C--You know, in this test the subjects have a longer time to work over the problem 
than they do in the others. I wonder if that’s contributing. The consequences may be 
different; did you hurry them along in that one? 

A 

S— On the Consequences Test we gave two situations and six minutes for each. 

We don’t have the correlation, since the tests were given at the same time to the two 
different groups . 

9 . ^ ® thinking now within the limits that Dr . Guilford has set up for testing times 
for the various tests, and I know on Controlled Associations they have a good long time 
to work this thing over. In contrast, in some of the other variables-- Plot Titles, for 
example -- you've only got a few minutes to write them down. I wonder if this is a 
factor entering into better predictability out of the associational fluency. 

S--On the Idea Classification Test, we had separate scores for obvious and for 
remote answers. I won't go into the operational definition and distinction. We found 
for the remote solutions that the research people were higher than the other groups, but 
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that there was no correlation with success within the function. However, for the obvious 
solutions, there was no difference between the functions whereas there was correlation 
with success within the research group. I would interpret all this to mean that you are 
selected on the remote responses to some extent, but then you are valued according to 
having a little common sense and the ability to reject the unappropriate original response. 

The productive thinking or Consequences Test did not give us much of anything, 
although there was some interesting material in it. It seemed to relate more to other 
functions than the ones we are interested in here. The breakdown that seemed to be 
important was in terms of a kind of content analysis of the responses along a dimension 
like the idea -things -people -economic dimension. Since it didn't relate to the groups 
that we are interested in, I’ll stop. 




37 






2n 



I- 



0 - 



-I- 



. 2 _i 



INDUCTIVE REASONING 



i n nr y u 

TYPE OF ENGINEERING 



I- 



0 - 



-I- 



- 2- 1 



ITME : IDEAS SCORE 



i n nr y i 

TYPE OF ENGINEERING 

38 



r2 




-I 



-0 



—I 



Yn 



L - 2 

Fig, 1 
-2 



-I 




-0 



— I 



YE 



-*2 



Pig. 2 



‘ERIC 



2 



PR|: TALKATIVENESS 



r2. 











l 





i n nr y u 

TYPE OF ENGINEERING 
PRh LIKING TO THINK 





Fig. 3 

r2 











2 i n nr y in 

TYPE OF ENGINEERING 









Fig. 4 






39 




PRI: TOLERANCE OF AMBIGUITY 




2 I II IF ¥ FI ¥E l ~ e 

TYPE OF ENGINEERING 

I Q 

PRI: MASCULINE VIGOR f 2 














■* i n nr y h m 4 

TYPE OF ENGINEERING P!g . 6 

40 






SOCIAL AND TECHNOLOGICAL DETERMINERS OF CREATIVITY 



Lindsey R. Harmon 
National Research Council 



S- -Creativity is a term which means many things -- and different things to dif- 
ferent people. Perhaps as a generalization, we could agree that it means the quality 
or ability to promote or effect creative activity. But such activity may be in any number 
of lines or modes . One can be creative artistically, administratively, pedagogic ally, 
or in the conduct of scientific research. Within the field of scientific research one might 
be most creative in constructing valuable hypotheses, in devising tests for these hypoth- 
eses, in constructing mathematical models, in executing experimental procedures, or 
perhaps in other ways. Creativity, even within a limited field, must then be a quality 
which is manifested in any number of ways. If we accept this as sound, does it not 
follow that creativity, if it is to be thought of as a characteristic of human beings, must 
be a very complex characteristic? Without making so bold as to estimate the factor 
structure of such a thing as general creativity, it would seem to be a good possibility 
that it is as complex as the test behavior which a few years ago we termed general in- 
telligence . 

If we are agreed that creativity is exceeding ly complex, perhaps it might be well 
to pause and see whether we can then agree on some line of attach which might structure 
the field. But instead, I am afraid that we must go, rather, in the opposite direction -- 
to suggest that creativity is an even more complex business than a many-factored ability. 
In brief, my thesis is that, if we are to think of creativity as an ability -complex at all, 
we must think of it not as something static, but as in continual flux, changing according 
to social and technological developments. This is, of course, a disquieting, not to say 
a disruptive idea. A complex but static thing is difficult enough to experiment with. A 
quality in a constant state of flux is exceedingly awkward to grasp, and even more hard 
to pin down in a controlled experiment. Such a concept requires justification, and you 
are entitled also to some indication of the possibilities of a research attach upon such 
an elusive concept. 

We have said, by way of definition, '*iat creativity is any quality of characteristic 
which facilitates the creative process. This is our basic definition, and yet it imme- 
diately requires a further definition -- that of the creative process. 

By the creative process is meant any process by which something new is produced -- 
an idea or an objec*, including a new form or arrangement of old elements. An essen- 
tial stage of this process is that the new form or gestalt must occur in the mind of some 
individual. The new combination, moreo\er, must meet some criterion of logic or es- 
thetics or both; not just any random combination of elements can be considered creative. 
Essentially, the requirement is that the new creation must contribute to the solution of 
some problem. The problem might be the production of an esthetic effect, the simpli- 
fication of a concept -structure, the education of a useful hypothesis, a means of testing 
an hypothesis, etc. The creative process, in short, is goal -directed. This goal will 
be determined by the interests and motivations of the person or persons engaged in the 
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creative process. That is, it may be some practical end, but it is not necessarily so. 

It may be knowledge for the sake of knowledge, or art for art’s sake. The creative 
processes with which we are most concerned at this point are those directed at the ad- 
vancement of science, but that in itself is of course an exceedingly complex and many- 
faceted objective. 

I have said that the new gestalt must at some point occur in the mind of an indi- 
vidual. This is not to imply that the creative process is the result sol *y of that indi- 
vidual’s efforts. No individual lives in a social vacuum, and the creative process may 
be largely individual. Or it may be highly determined by the investigator’s interactions 
with other people. We are concerned with all of the factors which determine whether 
the particular human being will bring about a new form or structure which facilitates 
the attainment of his goal. These factors relate to individual qualities or abilities, to 
technological developments, and to the social climate. Let us consider how these three 
factors may interact to facilitate or inhibit the creative process. 

Starting first with the individual human being, we can all agree, I am sure, that 
to act creatively requires that one be able to absorb a considerable fund of information 
and that he have a memory which can store this information and recall it selectively 
at the appropriate time. He must also be able to manipulate these ideas or memory 
traces in a disciplined and orderly fashion. Individuals differ constitutionally, we know, 
in their capacity for all of these functions -- the ability to form, retain, and manipulate 
ideas. This far, we are on the thoroughly familiar ground of standard psychometric 
formulation. But this is only one third of the picture. 

People vary also in their opportunities to acquire useful information and to prac- 
tice systematically the recall of learned information so that an orderly store of remem- 
bered elements is developed. They vary also in their opportunities for training in the 
various disciplines concerned with the manipulation of ideas. These differences in 
opportunity are largely socially conditioned. The social context varies not only in the 
opportunities it provides for training in intellectual skills, but also, and perhaps even 
more importantly, in the kind of motivation -development it encourages. An environ- 
ment may encourage innovation or conformity. It may abound, with people who are full 
of ideas and who tend to develop in the minds of growing youngsters the ideal of the 
scholarly or scientific life as a desirable goal. Or it may be barren of such influences. 
In addition to all of this, we know that individuals vary, too, in the extent to which 
they are responsive to the social environment. The mature, and not the young alone, 
are subject to such influences . We are much aware that an ’’atmosphere” or ' climate 
of opinion” may be either encouraging or inimical to innovation. In this whole area -- 
the social context --we are able at present, so far as lam aware, to deal with problems 
only descriptively, rather than experimentally orquantitatively. 

Creativity, we have said, is also conditioned technologically. We need only to 
consider such a technological advance as the development of the printing press to realize 
what tremendous differences technology makes in presenting to a potentially creative 
mind a wealth of useful ideas. Technology contributes in other ways, too. By freeing 
mankind from the primitive business of grubbing daily for the very means of subsistence, 
it has brought about opportunities for leisure during which the cultivation of intellectual 



disciplines is possible. From the earliest times of which we have any record, organ- 
ized society has provided for a small leisure class. But until the development of. the 
agricultural and industrial technology of the past century, this leisure class was nec- 
essarily limited to a very small segment of society, and not necessarily a segment 
which by position or motivation would tend to be innovators. Now modern communica- 
tions offer new vistas. For example, airplanes make possible such a conference as 
this, assembling people from thousands of miles apart within a few hours, and following 
the conference, returning them home with equal speed, and, we hope, with new sets of 
stimulating ideas gleaned from each other. A generation or two ago, travel for such a 
conference would have required two weeks at least of arduous effort. Electronic com- 
munications have only just begun to do what might be done, and what one day must be 
done to make it possible for human minds to bring into awareness at the proper time 
and manner the idea -combinations which result in new creations. Far from being sub- 
ject to controlled experiment, the technological aspects of creativity are as yet only be- 
ginning to be recognized. And meanwhile the technology which promises the greatest 
impact on creativity is in a rapid state of development. 

Up to this point, we have briefly considered the fact that individual, social, and 
technological factors all are important in the creative process. I have proposed, also, 
that we must consider these factors as changing or dynamic. By this I mean that differing 
patterns of human abilities may at various times, under varying social and technological 
conditions, contribute most to the creative process. As I have defined creativity as that 
ability -complex which promotes the creative process, it would follow that creativity it- 
self may or must be considered as in a state of evolution, as social forms and technolog- 
ical equipment change and develop. It will be valuable at this point, I believe, for us to 
consider the findings and formulations of a man who has been much concerned with crea- 
tivity. William Shockley, as director of transistor physics research at Bell Labs, has 
amply demonstrated his own creativeness. He has also been much concerned with the 
creativeness of the laboratory workers under his direction, and, more broadly, in that 
of laboratory scientists in general. In a recent paper delivered before the Operations 
Research Society of America, on the statistics of individual variations of productivity 
in research laboratories, he described his findings and set up two possible rationales 
to account for these findings. Both of these rationales are of interest, because both cer- 
tainly contain at least a germ of psychologicaltruth, and maybe very pregnant formulations. 

The basic fact regarding productivity which Dr. Shockley has documented, is that 
in respect to both patents granted and technical papers published, research workers vary 
much more widely than would be expected on the basis of a normal distribution concept 
of ability. In fact, the distribution of productivity is also highly skewed, following a log- 
arithmic function, which for brevity he refers to as a log-normal curve. (It must be 
stated at this point that, in regard to production of published papers, the research' of 
Wayne Dennis anticipated Shockley by some years). By such criteria as Shockley was 
able to adduce, these public evidences of productivity or creativity were fairly good in- 
dices of whatever one might ideally regard as true creativity. Satisfied that the log- 
normal distribution is a substantial representation of reality, Shockley turns then to 
rationales which would account for such a distribution. He considers two simplified 
models of the creative process, and develops from these models the empirically -dis- 
covered statistical distributions. 



Shockley's first model is based on the concept that a patentable invention requires 
a new combination of features and an appreciation of how this combination may l>e useful. 
He illustrates with the self-starter, which in his terms would require a new combination 
of four distinct ideas brought into awareness at one time. He then postulates an ability 
of the human mind which would allow the individual to be aware of "X" number of ideas 
and their relationships. People vary in this ability, so that they can simultaneously en- 
tertain 2, 3, 4, . . . n ideas. It would thus take a 4-idea man to invent the self-starter 
in the form Shockley conceives of it. But a 5-idea, 6-idea, or 7 -idea man could invent 
it much more easily, because they could hold the necessary four ideas along with one, 
two, or three irrelevant ideas. Their capacities would allow them, in fact, to consider 
the four necessary ideas in five, fifteen, or thirty times as many different ways as 
could the 4-idea man. An increase in brain-power of 50% can thus produce an increase 
in invention rate of 1500%. Shockley follows out the details of this system to some ex- 
tent, and shows that this kind of advantage increases very rapidly with increasing com- 
plexity of the problem. For example, for a 10 -idea invention, an 11 -idea man has an 
11 -fold advantage over the 10-idea man, so that a 10% increment in mental capacity 
produces an 1100% increase in output. He shows that a normal distribution of such an 
idea -capacity function would produce the empirically -discovered log-normal distribution 
of productivity. Whether this particular formulation is the correct one or not, Shockley 
feels, some such attribute must control productivity in a very sensitive way. 

His second rationale is based on the hypothesis that the interacting mental factors, 
of whatever nature, are of several different kinds, rather than several degrees of the 
same kind of ability. He cites a hypothetical list of abilities which might be involved, 
and assumes diat they may interact multiplicatively, whereas the usual psychometric 
formulations assume an additive kind of relationship. He shows how, by this reasoning, 
a relatively small variation of specific attributes can produce large variations in pro- 
ductivity, and that the log-normal distribution of productivity would be found. 

Although some obvious research possibilities are suggested by these rationales, 
such as study of multiplicative relationships between abilities as a predictor of a cri- 
terion of creativity, it is a quite different aspect of the problem with which I feel we 
should be primarily concerned. One can take either of these rationales of Shockley's 
and envision social or technological developments which would facilitate the functioning 
of his models and increase productivity enormously. Let us suppose, for example, 
that a 3 -idea man was working on the problem of a better way to start an auto engine. 

By hypothesis, he would be incapable of the invention of the self-starter because it re- 
quires four ideas simultaneously in awareness. He is blocked. The thing just won't 
jell. But assume that he is teamed with another 3-idea man who is working on some 
other problems of a related and contributory nature -- say small direct-current electric 
motors and clutch mechanisms for them. In the process of their labors, these men 
have already combined certain required elements into new functional wholes -- i.e. 
reduced the complexity of the problem from a 4 -idea one to a 3 -idea one or even to the 
proper combination of two of the newly-created syntheses. If they are proper and 
effectively teamed, they may well exchange their new, partially -formed ideas and might 
well hit upon the combination necessary. Whether they will or not depends a great deal 
on social and temperamental factors. I am sure we have all seen something of this kind 
happen in research teams, in conferences, staff meetings, etc. We have all witnessed 



groups that "team" effectively in such intellectual effort and others who do not. In brief, 
using Shockley’s first model, we can see ways in which 3-idea men can become a 4-idea 
or even possibly a 5 -idea team, in theory at least. 

Following Shockley's second rationale, which he favors as being closer to reality 
as he experiences it, we can again see how it would be possible to team people of var- 
ying talents in such a way as to produce greatly enhanced results. We know, as psychol- 
ogists, that people vary widely not only in single abilities, but in patterns of abilities. 
The multiplicative model is one in which any weak spot in an otherwise very effective 
combination would result in low productivity. People vary in their weak spots and in 
their positive strengths. If we can set up a team such that. the strengths of one man fill 
in the gaps in another’s abilities, we have a model with a strong combination of factors 
all across the board, and, theoretically, one which would lead to high productivity. It 
is immediately evident, of course, that one could never expect the coordination in a 
team that is possible within a single individual. The efficiency of such a synthetic group- 
ing of abilities would therefore be reduced considerably below what could be expected of 
one person with such a set of abilities. Nevertheless, the possibilities in this direction 
are most interesting, and deserve some research investigation. Among the many sorts 
of things that can readily be explored on a purely deductive basis are such matters as 
the upper boi ds of personnel statistics such as the possible frequencies of teams of 
two, three, or more persons with given levels of ability, assuming various degrees of 
intercorrelation among a given number of traits. These frequencies could be compared 
with the frequencies of such patterns within single individuals, given the same assump- 
tions about numbers and intercorrelations of traits. Research should be done, also, on 
factors affecting team effectiveness, sources of friction within teams md means of re- 
ducing them. 

The matter of teamwork in scientific research was touched upon recently by one 
of this country's senior scientists -- Vannevar Bush. Writing in the August Atlantic 
Monthly, in an article chiefly devoted to the motivations to scientific and scholarly work, 
Bush touches rather briefly on the question of teamwork in research and also harks back 
to a previous theme of his, and to what I have proposed as the third main factor in the evo - 
lution of creativity - - the development of technology that will assist man's thinking pro- 
cesses in much the same degree as it has in the past his perceptual processes. What the 
microscope, X-rays, sonar, and a host of other electrical and chemical processes have 
done for our ability to perceive our physical environment, technology may also be har- 
nessed to do for our ability to think. The development of high -speed automatic electronic 
computers has been the most spectacular development along this line to date. But these 
computers have attacked only those aspects of our thinking process which are conceptually 
the simplest, and serve us well because of great speed, and, more recently, great mem- 
ory capacity. But their operations are in the most elementary arithmetic -- the binary 
system - -and they are able to attack more complex problems only by brute force methods 
and the most extensive direction by highly skilled mathematicians and engineers . 

Bush has given us a vision of something else — something with far greater poten- 
tial for our problem of creativity. Ten years earlier, writing in the same magazine, 
in a fabulous article entitled "As We May Think, " Bush sketched the possibilities for 
a number of possible mechanical and electronic aids to thinking. Among them were 
devices to perform most of the routine operations of algebra, leaving for the mathema- 
tician the truly inventive or creative aspects, by freeing his mind from the merely 



mechanical manipulations that go by well defined rules. Thus freed, he can operate at 
higher speed, perhaps often catching "on the fly” insights which might be lost because 
of their fleeting character, when the mathematician is encumbered with the burdensome 
routine of setting down operations and transformations by paper and pencil. And inci- 
dentally, the machine would probably not only be easier and faster, but would make 
fewer errors in these routines . 

But it is not mathematical machines that are Bush’s main interest. His main 
concern in 1945, and in 1955 also, is the exponentially accelerating accumulation of 
new scientific knowledge that nobody is able to read and digest. We have multiplied 
our means of producing new science. We have lagged seriously in our means of storing 
it effectively, and most particularly in our means of consulting the record. What Bush 
proposed, in brief, is an electro -mechanical aid to the memory process, which will sup- 
plement our relatively feeble and inaccurate cerebral memories . What he is talking 
about is at this stage but a dream. However, Bush is a remarkably practical dreamer, 
and what he envisions is within the realm of the realizable, if the technological effort 
is expended on developing it. I cannot now go into details with regard to his machine. 
Suffice it to say that it involves ultra-fine microfilm processes and electronic and me- 
chanical means for rapid consultation of the microfilm memory, following processes 
of "association of ideas" just as does the human mind, rather than merely the hierar- 
chial categorizing system used for library reference purposes. The machine, which 
Bush dubs "Memex, " has as its main advantage over the human mind the copiousness 
of a library of a million books and the stability of a printed page, rather than the grad- 
ual decay of human memory traces. Bush envisions the development, when such ma- 
chines are available, of new intellectual specialists, whose job it will be to blaze new 
trails of thought through the vast jungle of our accumulated and accumulating knowledge, 
and the ready exchange among scholars of microfilm records of these new trails. One 
function of such a trail, of course, would be much the same as a special review of the 
literature, of the kind we are familiar with in "Psychological Bulletin", for example* 

But trails could be blazed, too, which would cross the lines of various disciplines and 
sub -disciplines, helping to knit together the specializations which have for a long time 
past divided and subdivided the fields of human knowledge. It is in such cross-disci- 
plinary areas, we know, that much of the most creative work has been done. 

I cannot in this brief time do justice to the possibilities of Bush’s brain-child, but 
my concern now is rather with an aspect which Bush has barely touched upon. He en- 
visioned exchange of the contents of the Memex memory 1 y trading about of micro- 
film records . One need add to this only the possibilities of television circuitry to open 
up great vistas which are of particular concern to our topic today. Consider, for a 
moment, these possibilities. Memex is the mind of the scholar or scientists - -the 
memory part of his mind, at least. What Memex does is to present a visual picture of 
its contents on something like a television screen. The old Chinese knew that a picture 
is worth a thousand words. Consider now the speed with which we can perceive pictures -- 
pictures of objects, printed pages, formulae, or motion pictures of processes - -as com- 
pared with the speed with which we can perceive words through our ears. Suppose, then, 
that it is possible through proper circuits, for one person to tune in on another’s Memex 
in operation. In effect, one can "read the microfilm mind" of the other. A sort of 
"thought transference" is accomplished, and at a speed, and with a clarity and stability 
of detail that far outdistances any present means of communication. 
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The possibilities of such technological developments stagger the imagination. 

At present, a lecturer has the advantage of a public -address system in speaking to a 
student audience . More students can hear him clearly - - perhaps a thousand or two 
rather than the maximum of hundreds and the usual dozens of previous generations . 

But he still is harnessed to the speed of oral speech, supplemented, it is true, with 
slides and films from time to time. But suppose that they could simultaneously watch 
a screen on which is flashed the equivalent of the mental images of the lecturer! (One 
might surmise that the thought processes of some lecturers would require close cen- 
sorship if this were done --it could go too far!) But our main concern at this point is 
not with the teaching possibilities of such a medium, but its potentialities for facilita- 
ting the creative process. 

We are in a position now, I think, to bring together the three main aspects of 
creativity of which I have spoken -- the matter of individual capacities, the possibilities 
for teams of individuals whose abilities are complementary, and the possibilities for 
technological developments which might aid their mental processes . 

If we can set up teams of people with special skills, aided by electronic devices 
to facilitate application of these skills, and provide them with interpersonal communi- 
cation which has a speed, clarity, and precision that outdistances our present modes 
by a factor of two, or ten, or a hundred, we have taken vast strides to overcome the 
limitations of minds that can maintain in awareness only a finite and small number of 
ideas, or of minds with severely circumscribed patterns of abilities. Something like 
a collective mind can be seen as an. emergent possibility, by simple projection of social 
modes and technological devices which we already are able to envision, if not yet able 
to build. These devices can of ccurse, greatly aid the individual scientist, can over- • 
come some individual limitations or handicaps, and provide new modes of publication 
of his ideas and inventions to the world, thus breaking down certain present barriers 
to creativeness. But certainly by far the greatest possibilities lie in the area of teams 
of scientists in instantaneous and intimate communication, even over vast geographical 
distances . 

We have already anticipated that problems would arise in the effective exploitation 
of these team developments. Not only will teams have to be selected for optimum skill 
and knowledge combinations, but also from the standpoint of tempermental compatibility 
and similar value systems. For in the output of such an electronically -linked team there 
could be no question about individual credit. The motivational system of the team mem- 
bers must be closely tuned to team credit. The functioning of the team is everything, 
and unconscious barriers to communication would be the most destructive kind of sand 
in the gears, blocking the truly creative possibilities of the team’s functioning. But if 
such emotional problems can be worked out, if teams can be trained through some new 
developments of group dynamics to function in the intellectual sphere as smoothly as 
the Harlem Globetrotters function in basketball, the creative possibilities are quite be- 
yond our feeble imaginings at this point. They extend not only in the direction of greater 
speed and wider syntheses of new ideas in the most advanced area by the greatest minds, 
but in the direction of far wider participation in Me scientific creative process by people 
whose patterns of abilities and disabilities now quite effectively shut them off from par- 
ticipation in truly creative work. 
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At the risk of redundancy, let me now restate the central thought of this discourse. 
We are concerned with creativity. We seek to discover through research how scientific 
creativity may be increased. We have tended in the past to think of creativity as largely 
an individual matter -- as a trait or trait complex somehow residing within the individual. 
However true this may have been in the ^ast, it appears that we are now entering an era 
in which it may no longer be true, or at least not exclusively true . We must concern 
ourselves, in research on the question of creativity, with social and technological de- 
velopments. These developments offer great possibilities of creative activity in people 
whose abilities would otherwise not be adequate for creativity. We must, in short, deal 
with creativity as an evolving phenomenon. This greatly complicates our research prob- 
lems. What might have been anchor points for our research now become a state of flux. 
There are no fixed stars. Like the astronomers, we must consider a universe in which 
everything is moving, and moving at varying and largely undetermined rates of speed. 

If within our lifetime we can arrive at formulations even approaching in their concep- 
tual power those of the astronomers, we will have done a job beyond what we have any 
research to hope for now. 
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C'-This Memex thing is perhaps not so far off as we might think, at least in its 
elementary stages . One company is currently working on a revised kind of microcard 
or microfilm device, using very small plastic tabs revolving on circular hoops and ca- 
pable of being coded in such a way that they can be mechanically extracted very rapidly . 
A very sizeable library could be stored in one room without any difficulty whatsoever, 
and searched in a relatively limited time. Now, obviously this is a far cry from the 
sort of speed of search that could be incorporated in some of the electronic gadgets if 
you could get a big enough memory. But as a first step in this general direction, its 
going to be much better than anything now available, such as microfilm on reels where 
you must run through a whole reel in order to get out one or two items. This micro - 
card is a present reality. With a little more development, it will make possible a li- 
brary of 10, 000 volumes for everybody quite easily. 



C--You know, I wish that some of us would stop developing memory -storage de<- 
vices and try to develop forgetting machines. Because one of the real problems is that 
people can t forget enough to be open to new kinds of experiences . If you could get a 
selective forgetter it would really be helpful. 



Q--Why stop with a memory machine? Why not let the machine do part of your 
creative work too? I understand there’s a machine for creating melodies out of tones 
by casting chance combinations, some of which are melodies. 

C--Some of the difficulties are the contrast between the temperaments of psychol- 
ogists and engineers! 



C--I have the feeling that we need to do something better than, say, Psychological 
Abstracts -- that we need to organize and store and transmit much more detailed infor- 
mation. Somehow, I always have the feeling of "d^j'a vu" at conferences like this. Some- 
time in the past twenty years I’ve heard or read of all this experimentation before, only 
changed a little bit here and there. It's the non -cumulative nature of research that 
bothers me. Each researcher starts all over again with a slightly different set of items, 
and does another factor analysis. If only we could have a machine that would store 
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items and factor analyses, or even sub-tests, then we could pull out parts systematic- 
ally whenever we design a new research project. Then the results could be fed back in, 
and become cumulative, so that eventually we would have a large enough and systematic 
enough store of data to know where we stand. 

C--I think there's a reason why one researcher doesn't have and use another's 
items and tests, and it's not a mechanical reason. It's the demand for the surrender 
of individual narcissim. That's the essential thing that impresses me about this vision 
of the world of the future . 

C--One of the hopes of this conference is that we could bring together information 
about different researches with the ' ame kinds of material, and say what is true about 
such-and-such a test, and under what conditions. We could then set up a series of hy- 
potheses and be dble to test them under systematically varied conditions. We could thus 
state what are the limits of the various hypotheses, and what contrasting hypotheses 
could explain the same phenomena, as for instance Saunders' findings with his mathe- 
matical induction test. To my mind, that's a pretty good mathematics test, and it's 
understandable why engineers should do well on it. But we need to be able to formulate 
specific hypotheses which would distinguish between a mathematics achievement test 
and something quite different. 

C--But, taking that same test, we know that at Least ten persons have used variants 
of it, with different subject populations. But no one can now bring all these data together 
and show how they bear on the problem. We are only vaguely aware that something like 
that test has been used here and there. We need some means of making these data ac- 
cumulate at one point. 

-Yes, but every time you set up such a system, don't you exclude other possi- 
bilities? 

C — No, it's just a memory device. It's just for bringing together all the relevant 
data when you make up your new test. You could then take a good look at it and go on 
from there. It frees you from the necessity of repeating errors, and of making small 
but meaningless variations that prevent your data from being cumulative. 

C--Well, it's clear that increasing degrees of organization will increase efficiency 
and power up to a point, and thereby also creativity; but the danger is that you lose to 
some extent the possibility of newness as you standardize. It's a minimax problem; you 
want to maximize power and minimize restrictions on creativeness. You want to be sure 
that you are in fact maximizing freedom, and not inadvertantly reducing it by organiza- 
tion in the interests of power. 

jC - -Psychologists are clearly not at the point now where one person can organize 
the work of a large number of other people to the extent that freedom is restricted. We 
are too autonomous. It does seem that we could profit by being able to replicate exper- 
iments. As someone said earlier with regard to the mathematical induction test, it's 
very important to determine whether it is mathematical induction and not mathematical 
achievement in another form. So too with the work by Wilson in the Portland schools. 

If other researchers could apply some of those same tests in their studies of person- 
ality with other populations, we could add greatly to our knowledge. 

C~-Possibly one cf the reasons why we feel we're not building enough, that our 
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data aren't cumulative, is that we tend to be too empirical in our research — we don’t 
look at this material to abstract the laws, and to assess and hiild in terms of the laws. 

I hope that this conference will help us find places where we have communalities. If 
we can get as the laws, our new experiments could tackle them, rather than just throw 
in a few new items on the basis of face validity. It's when we do that that we destroy 
the comparability of our data. 

C--One of the big issues is the extent to which we are going to be practical, and 
to wl X extent we are going to do basic research that may have no immediate practical 
value. If it is immediate practicality that we want, we should operate more nearly on 
face validity, and on empirical results. But if we try to do some more basic research, 
we can begin to look for laws and really go off into the blue in ways that wouldn't other- 
wise be justified. 

C--I think that some really fundamental questions have been raised here with re- 
gard to what we are measuring -- such things as that question of mathematical induction. 
If we could formulate and share some of our really critical hypotheses, we could know 
in what directions we are going, and try to build on each other’s work, rather than each 
going off in his own direction. 

C--If the group could reach a consensus in identifying some of the critical prob- 
lems in which research is needed, and if enough individuals are interested in doing re- 
search with relation to these critical problems, we might provide some opportunity 
later to bring this research together, so that we could really have a coordinated attack 
on these questions . At present all these experiments are too scattered, the data too 
unorganized, so that we can't see patterns in the data which might suggest laws or hy- 
potheses that have significant generality. 

C> -But only single brains can think.. In all of this, I was reminded of Spinoza's 
statement about freedom of thought being an indefeasible natural right because every- 
one has a brain inside his own skull and there isn't going to be any thought transference. 
This kind of synthesis can come about if all the information can be pulled together so 
that one person can think it. But I don't think that groups ever do this; that is, the best 
you can do is exchange information. 

C^-You can exchange information in a pattern or context, and *n such time re- 
lationships as to make a significant synthesis of ideas more probable within the individ- 
ual. We might recur here again to the idea of selective forgetting. If you have too many 
things in mind, you've got to forget some in order to see the pattern in the data. The 
group situation in which iniormation is exchanged may also be important --it can stim- 
ulate or inhibit absorption and manipulation of information. If you have too big a group, 
or too much information at one time, you can't digest it. I think there’s a lot to be done 
in pulling together groups that can be creative because of the non -overlapping character 
of their contributions . 

C--I find it very valuable to look down another person's perspectives and to cor- 
rect my own deficiencies of insight simply by the suggestiveness of another point of view. 
I wonder if that might not work in this situation. 

C --Let’s take a specific problem, and see where it might lead. In Saunders' 
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material there was a measure of masculinity -feminity. I question whether that is really 
what was being measured, and believe that we could get into a very fascinating theoret- 
ical discussion on this . If we do, we might start to define some of the basic parameters 
amongst ourselves, so that we could build on this very problem. For example, it strikes 
me that we*re probably even perpetrating a social evil by calling it masculinity-feminity, 
because people have an unfortunate reaction to the term. Some of the items, on the other 
hand, suggested to me that what was really being measured was "acceptance of feeling" 
more than ieminity. Now, with the group mentioned, we have a beautiful combination of 
characteristics: high on number series and high on acceptance of feeling. Acceptance of 
feeling in combination with a high rationality factor Suggests a beautiful complexity of 
personality. Here, I think, we would get into basic theoretical issues where we would 
know what we were building on. This is just an illustration of what I had in mind. 

C- -Could one of the outcomes of this conference be a trichotomization of variables 
into cognitive, personality, and group aspects of the problem, with an announcement of 
crucial problems or crucial variables that should be studied within each of these areas? 
And, cutting across these, combinations of variables that we could define clearly, such 
as the number-series and personality variables just mentioned? I would like a list of 
these things that I could sandwich in my research where possible. 

C— We need something else, too, which relates tr the resources of various invest- 
igators. I'm not set up with equipment for high-powered statistics such as factor analysis 
or pattern analysis. We might hope that by announcing these things, with some indica- 
tion of the magnitude of research evidence required, people could pick and choose, de- 
pending on what their resources are . 

C— Time has been left at the end of this conference to go into more detail in these 
kinds of things. 
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RELATIONSHIPS BETWEEN MEASURES OF 
SCIENTIFIC PERFORMANCE AND OTHER VARIABLES 

Donald C. Pelz 
University of Michigan 



S--I would like to talk about some relationships between measures of scientific 
performance and other measures of motivation and the environment. 

The Institute for Social Research at the University of Michigan has conducted 
two studies that bear on scientific research. One of these was done at the National 
Institutes of Health by Davis, Mellinger, Pelz, and Baumgartel. We restricted our 
performance measurements there to about 300 "intramural" scientists, that is, people 
doing research work in the laboratories at Bethesda, Maryland; we excluded people 
with professional training who were largely working in "extramural" programs such 
as research grants. The second study is one that we did for a "Survey of Physiolo- 
gical Sciences" which was initiated by the American Physiological Society, and which 
was supported by money from the National Science Foundation. 

Performance Measures 

At the National Institutes of Health, we obtained a single measure of overall 
scientific performance. The scientists at first resisted strongly. When we men- 
tioned our interest in measuring scientific productivity, they claimed that it could 
not be done; but eventually, as we developed our plans for an attitude study, they 
became increasingly dissatisfied with simply studying "happiness, " and insisted that 
we include a measure of scientific performance. We didn’t call it creativity, but 
used "performance" as the most neutral term we could think of. We definitely a- 
voided calling it "productivity. " They were quice firm that no quantitative meas- 
ures such as number of papers could capture the subtleties of real scientific achieve- 
ment. 



What we worked out with them was a system of panels. The 300 scientists can 
be grouped in two ways so that the people within a given grouping are reasonably fam- 
iliar with each other's work. One kind of grouping is in terms of administrative units, 
usually the laboratories. There are some 20 laboratories, interdisciplinary in nature, 
each working on a certain type of problem. The laboratories range in size from 10 to 
50 scientists. The second type of grouping is in terms of scientific disciplines, cutting 
across laboratories. For example, biochemistry is found in several laboratories, and 
the people in that field keep up reasonably well on each other’s work. 

♦ 

Having formed these two sets of groupings, we asked for nominations of judges 
within each - -people who were reputed to be themselves capable in the area, to have 
been around long enough to be familiar with the work of other people in the same 
grouping, and to be relatively objective in their ability to assess other people. Each 
nominee was asked for other nominations . By a process of successive consensus, we 
finally weeded out from two to eight judges within each of the groupings. It is impor- 
tant to note that each individual was judged by scientific colleagues as well as supervi- 
sors, not only in his laboratory but also in his field of scientific work. The resulting 
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judgments were more by peers than by superiors. 

We then gave each judge a list of the individuals in a particular grouping and had 
him divide the list into two or more categories from higher to lower . We avoided a 
rating system. The scientists would not tolerate ratings because of their negative ex- 
periences with Federal Civil Service ratings, efficiency ratings, etc. We asked the 
judges to make as many or as few distinctions as occurred to them naturally among 
these individuals. Also, we requested only a single overall assessment, rather than 
judgment of separate qualities. We asked them to imagine themselves on a panel ad- 
vising on research grants; they had a certain sum of money to give for research in this 
area; which individuals would they recommend as being "best equipped to turn out the 
best possible research within the next few years'*? What we were obtaining was their 
judgments of the individual's current potential to do outstanding research. This was 
called "overall scientific performance. " 

In all, 72 persons served as jud^ ;s - -about one -fifth of the total population. We 
eliminated a few judgments whose assessments did not correlate with those of other 
judges. We eliminated a few scientists on whom judgments of different assessors were 
inconsistent. By and large, ws lost not more than 5% of our total population for these 
reasons. 

Judgments were translated into numerical scores based on percentiles, and an 
average score computed. For about 75% of the individuals there were two ratings, one 
in the laboratory and one in the discipline. The median correlation between them was 
.84. The two scores were averaged to obtain a single score for each scientist. 

In the study that was done for the Survey of Physiological Sciences by Lieberman 
and Meltzer, the committee sponsoring the survey attempted to get a complete listing 
of the total population of physiologists. Membership lists of three major physiological 
societies were obtained, as well as fringe people in other societies, such as physiological 
psychologists. Some 7, 500 names were thus assembled. Each was sent a questionnaire 
after suitable advance publicity. The return was fairly good --about 75%. A 10% sample 
of non -respondents failed to reveal any substantial bias for non -responses . 

The measure of scientific accomplishment in this study was a relatively simple 
one. For some 1,800 of the population who were members of the three major societies, 
a search was made through the Annual Reviews of Physiology for three years; the number 
of times that each individual was mentioned in the Annual Reviews was used as a measure 

I 

of output during this three -year period. To some extent, this is a measure of publication; 
there is a correlation of about . 50 between the number of papers published during the inter- 
val and the number of citations. Some people are cited who haven't published at all during 
this period. On the other hand, whereas the mean number of papers published is about 6, 
the mean number of citations is only 3. Hence, a selective process is operating, presum- 
ably in terms of quality. 

I should point out, by the way, that in both the NIH and the physiology study the sub- 
jects are working in an academic atmosphere. Two -thirds of the physiologists work in 
universities, while relatively few work in either industry or government. As far as NIH 
is concerned, the atmosphere there is quite an academic one. There is a high degree of 
individual autonomy, and even for a government laboratory they have a reputation of 
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providing freedopi for the individual worker. I'd like to return to this point later on, 
because one of the things that I’d like to emphasize is that the kind of atmosphere or en 
vironment in which the individual is working may make a considerable difference in the 
results. The predictions you can make for industrial researchers may not hold true in 
an academic setting. 



Performance and Values 

Regarding relationships between performance and other variables, let us consider 
first of all some data in the motivational area. On a questionnaire given to physiologists, 
we asked what kinds of factors they regarded as important in a job. About ten of these 
were listed; the individual's subjective importance attached to a given factor was used 
as an indication of the individual's motivation toward that goal. 

For the NIH people, a simple cluster analysis of intercorrelations among these 
ratings of importance showed some interesting and suggestive patterns . First of all, 
there seemed to be one cluster which we called "science orientation. " This consisted 
of three items: importance attributed to the opportunity to contribute to basic scientific 
knowledge, importance attached to freedom for originality, and importance attached to 
use of present skills and abilities. These three intercor related and did not correlate 
with the other items. 

The use of these particular items stemmed from a previous study that our Insti- 
tute had done in the Office of Naval Research itself. A Rockefeller fellow named Dwaine 
Marvick did an analysis in terms of career patterns of the people at ONR. He defined 
three types. One he called the "specialists. " These are the people who look to their 
professional colleagues for approval and who see their natural home as the university. 

A second type he called the "institutionalists . " These are people who see their career 
patterns in terms of remaining with a particular hierarchy, either military or Civil 
Service. A third type, or "hybrid, " had some characteristics of both the others. 
Marvick found some substantial differences between people whose career orientation is 
towards a scientific specialty, and those whose career orientation is toward a particular 
institution. 

At NIH we wanted to see if the same two types would emerge, and in fact they did. 
In addition to the "science orientation, " we found an "institutionalist" pattern at NIH — 
people stressed things such as having an important job, associating with high level 
people, and belonging to an organization having prestige in the lay community. These 
three items constitute another cluster of status -oriented or prestige -oriented items . 

Incidentally, the indices of "science" and "institutional" orientation are uncorre- 
lated, not negatively related as you might expect. A scientist can be high or low in 
either or both. 

C--It reminds me of Carter's work where he cranked out two factors in group 
studies, although not with scientists. One factor was goal facilitation which might 
typify tjhe first cluster, and the second was individual prominence where goal facili- 
tation didn't matter. The idea was to advance one's individuality upon the group. 

i 

C--This would not necessarily’mean that the prestige -oriented people are all in 
the non -academic institutions, and the others are all in academic institutions. You get 
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plenty of this in the academic institutions. 



S- -True. But in general the prestige -oriented persons do not see themselves in 
an academic setting. We asked, "If you were to leave NIH, what type of situation would 
you most prefer to be in: a university, another government organization, industry, or 
something else?" Among those who were high in science orientation but low in institu- 
tional, about 85% saw their natural home as the university; whereas among the people 
at the opposite position --high on institutional orientation but low' on science - -only 42% 
saw their natural home to be the university. 

NIH, I think that your institutional group would tend to be those who had 
been in the service for a longer period of time. You’ve got the career Public Health 
Service people who would be in that group, I think. 

S--Tbac ties in with another point. We also asked, "How important is it to you 
to contribute to the nation’s health?" This correlated with the institutional cluster, but 
did not correlate with the science cluster. People who are oriented toward practical 
accomplishments in health are also the ones who are oriented toward prestige, rather 
than toward scientific knowledge. 

Next, we ran several tests to see how these motivational items correlated with 
the performance measures. The science orientation correlates quite well with scien- 
tific performance; the institutional motivation does not. There seems to be no relation- 
ship between the strength of the man’s institutional motivation and the way he is seen as 
performing by his scientific colleagues . 

Now, I suspect that our measure of scientific performance reflects "basic” re- 
search performance rather than "applied. ’’ We got the impression that the people who 
were doing the applied research were looked down on by the "longhair" scientists who 
in general constituted our assessing panels. For example, M.D. ’s tended to receive 
lower scores than Ph.D. ’s. If a different set of judges had been used, with a more "ap- 
plied" concept as to what accomplishment means, institutional orientation might have 
been related to their assessments. 

2" “You have a fair mixture of clinicians in there, though, and I think that they 
would tend to mix the broth a bit. The M. D. ’s — how many of those are there? 

S— Among the professional staff, 31% were M.D. ’s and 43% were Ph.D. ’s or 
Sc.D. ’s. 

Q- -Would the M. D. *s be on the panels? 

S--Yes, some of them would -- I don't know how many. But unless they were re- 
garded as competent researchers they would not be nominated. 

Let me go on. I had a chance to analyze the physiology data, and a similar result 
emerged. Using a cluster of three "science -oriented" items which were parallel to the 
three used at NIH, we got exactly the same thing - quite a high relationship between 
science orientation and number of citations, but no relationship between other motiva- 
tional items and citations. For example, the individual ’s emphasis on opportunity for 
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"advancement in my profession" was unrelated to his output. 

At a conference that was held at Ann Arbor early this spring, Herbert Shepard 
from MIT reported a rather interesting result which to some extent conflicts with the 
findings from these two studies. He secured the cooperation of 21 industrial labora- 
tories in which executives indicated high and low producing teams. He then gave a 
projective picture in which a stout man with a bald head and watch chain, sitting back 
in his swivel chair --an obvious executive type --is talking to a young man in a white 
coat and crew cut. The older man is saying, "We have fine opportunities in our depart- 
ment. Why, in the last year, three of our best men have (blank). " The respond- 

ent fills in the blank. Shepard scored the answers in several categories, of which two 
correspond roughly to our institutional and scientific orientation. In one case, the re- 
spondent sees the opportunity as being "up and out. " That is, he becomes head of the 
sales department or vice-president in charge of production; he moves "up and out" into 
the administrative hierarchy. The second response is that he is promoted within the 
research set-up; he simply gets better facilities to keep on doing research.' These two 
responses seem to parallel our "institutional" and "science" orientations, respectively. 

Now, Shepard found no difference in the relative frequency of these two responses 
between the teams that research execute 2 s regarded as high or low in several ratings 
of "productivity" or "creativity. " There seemed to be no relationship between orienta- 
tion and performance. 

We have wondered, therefore, whether in an industrial atmosphere it may not be 
necessary to have both scientific and institutional motivation in order to succeed in the 
eyes of the company. It may not be a question of one being strong, but rather of both 
being strong. 

C--It’s a semantic confusion when you use different operations for measuring 
these things. We call the kind of thing you measured at NIH either interests or value -- 
when a person consistently states that he values something. When you use the projective 
device, you’re not asking him that; you're asking him to project something. And we find 
that these two measures in the same person have essentially a very low order correlation. 
So it may be that the difference is simply due to the fact that you have different measure- 
ments. In one case, you’re really asking the person to commit himself; and whenever 
the self enters intp it at the conscious level, all kinds of self -valuing factors enter in. 

C— Since the Ann Arbor meeting, I’ve had a little more time to think about those 
things. What we’ve started to do in our stuff is to separate the institutionalist who iden- 
tifies with the values of the organization from the institutionalist who seems to be a really 
more dependent kind of person, whose ego income comes from all kinds of external af- 
filiations. He’s an other -directed kind of individuaL You really have two kinds of insti- 
tutionalists. 



S--Yes, I think that it would be a mistake to assume a one -or -two -dimensional con- 
struct here. In my conversations with industrial researchers, one of the obvious sources 
of institutional motivation was the feeling that they were doing something useful for the 
company; and this seemed qualitatively different from a striving for personal prestige. 




57 




However, although these two aspects are qualitatively distinct, my hunch is that they 
will be moderately correlated; possibly both of these components may be needed for 
success in the industrial setting. Obviously, we need to do more exploring to see what 
are the different components in interests or values. These different components may 
relate in quite distinct ways to performance. 

C--I've been trying to ’’psych-out” why the high institutionalist feels that he wants 
to improve the nation's health, whereas the science -oriented person doesn't necessarily 
feel that way. It seems to me that a socially approved answer, one that the institution- 
alist would be most likely to give, would be: "Yes, what I’m doing relates to the improve- 
ment of the national health. " But the response of the science -oriented person would be, 
"This is to laugh. Here, you punk, day to day I’m involved in work that is designed to 
do that. " It is an ironic question to ask of a man who’s devoting his full time to a piece 
of research, however narrow, for improving the national health. 

S--I don’t necessarily agree. Most of them regard themselves as doing basic re- 
search; the fact that it contributes to the nation's health is a way to get support, but 
many cf them don't really care about the ultimate applications. 

_C--I wouldn't deny them that much altruism. I think it’s a case of immersement-- 
they are so immersed in a basic component of the total picture that they feel like throw- 
ing the interrogator out of the window when he comes along with that sort of a question. 

C--I think he's really getting two different personality types. The individual who 
says he is going to contribute to public health has probably got higher dependent needs 
than the individual who's on the science side. The latter is more rational, more de- 
tached from early parent -child relationships than the institutionalist. The institution- 
alist is still too dependent on early parent -child relationships. 

C--You have in one case a fundamental commitment to the world of ideation, and 
in the other case a fundamental commitment to the v/orld of human interaction. 

C--I should think these institutional people would break into two groups - -people 
who are dependent, and people who are aggressive, that is, people who want to dominate. 

S--It’s an interesting idea, and one worth exploring. , 

Performance and Supervise 

Next, I’d like to discuss some of the social factors in the environment related to 
performance. Let’s consider supervision. What kind of supervision is correlated with 
high or low scientific performance? Our results indicate that we must be cautious about 
making broad generalizations . (My material on supervision is based on unpublished data. ) 
For one thing, NIH data suggest that the relatively younger scientists may show different 
results than the more mature scientists . By younger is meant those in Civil Service 
grades GS-9 and U, and by more mature those in grades 12 and up. The younger people, 
for example, show a definite relationship between scientific performance and a factor 
we called intensity of interaction with the supervisor; whereas, for the senior people, 
there is no relationship between their performance and intensity of interaction with the 
immediate chief. 

<Q--Is this amiable intensity or any kind of intensity? 
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^--Operationally, it was defined by an index of three items obtained from ques- 
tionnaires. One is the respondent's report of sheer frequency of contact. A second 
item is the respondent's estimate of how much the supervisor knows abdut the respond- 
ent’s work. The third item is an estimate of the degree to which the respondent could 
influence the supervisor’s decisions affecting his work. These three intercorr elated 
moderately well and were combined into a single index. 

C--Of course, you must realize that a man who is a grade 9 to 11 on the books 
is under supervision, and that will predetermine the amount of interaction that goes 
on. 



S--That is true; and that is why we separated the younger from the older group, 
because the younger group does, in fact, get a lot more supervision: they make many 
fewer decisions on their own. The important thing is that even within this relatively 
homogeneous group of people who get a lot of supervision, the more they get the better . 

Q--Won’t the supervision of the GS-12's and up be concerned more with adminis- 
trative matters than scientific? And might this not just be the reason why you get a 
correlation in one case and not in the other? 

S --Perhaps so. All I'm saying is let's not lump all scientists together, and assume 
that you can establish the same rules for the relatively younger people as for the older 
people. 

I also suspect that you would get different results depending upon the kind of 
task to be done. At NIH, there is very little that can be called teamwork. The actual 
groups were usually two or Ihree men, collaborators who worked together on a personal 
basis to write a joint paper. I suspect that in industrial settings where you have a gen- 
uine collaborative effort, you may need a different kind of supervision. 

Q - -Is supervision really the word that describes this? It is almost solicitude, isn’t 
it, for the younger man? It isn’t that he is being directed so much. 

S--We did a further two -variable analysis which bears on that. One variable was 
interaction with the chief, which I’ve just described; a second was the extent to which 
the younger man has the opportunity to make decisions concerning his research problems „ 
We found the highest performance where there is a lot of interaction, but where, at the 
same time, the younger urn has some freedom to make decisions. That would bear 
out your comment; the chief is apparently stimulating and encouraging the younger man, 
but does not actually direct him. 

One other interesting point emerged, concerning group identification. In some of 
our industrial studies, the higher producing groups were those who had a stronger sense 
of group pride and belongingness . Would the same results hold in a scientific setting? 
Again there was no simple answer; there was no simple tendency for high -producing 
scientists to have a stronger identification wiJi their immediate group. The relationship, 
rather, depended on who was head of the group. If the scientists had a strong sense of 
belonging to the immediate group and the chief of that group was a relatively competent 
person, then the subordinates’ performance was high. But if the scientists had high 
identification with the immediate group and the chief was a relatively mediocre man, 
their performance ;vis low. 
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C - “It seems to me that high identification with a problem rather ‘'ban a group 
would relate to performance. 

Performance and Relations with Colleagues 

S — Let me proceed with one other area, relations with colleagues. On the original 
questionnaire we administered a kind of sociometric instrument in which scientists in- 
dicated which individuals were most significant for their work. For each respondent 
we took his five most significant non -supervisory colleagues and obtained die average 
amount of contact between the respondent and these five colleagues. A second dimen- 
sion we used was the extent to which the respondent was similar or dissimilar to the 
colleague with whom he had contact. We measured similarity in several different ways. 
One was in terms of the values described earlier. If the respondent was a science - 
oriented man, and his five colleagues were also science -oriented people, we gave him 
a high score of similarity. If he was a science -oriented man whose colleagues were 
mainly institutionalists, we gave him a low score in similarity. The findings were as 
follows (this is unpublished data): For those scientists who are very similar to their 
colleagues, having contact only once or twice a week is the point of maximum perform- 
ance; more frequent contact than that is associated with a decline in performance. But 
if the scientist associates with people who are quite different from him in terms of 
values, then daily contact is associated with the highest performance . We find parallel 
results with two or three other ways of measuring similarity. Thus, contact alone is 
not related to performance, and similarity alone is not related; but when you put these 
two things together relationships appear. 

One interpretation is that if a scientist is thrown in with a group of dissimilar 
people, he needs a lot of contact in order to bridge the communication gap. Whereas, 
if he is thrown in with like-minded colleagues he doens’t need so much contact; too 
much group activity, in fact, may be a distraction in this individualistic setting. On 
the other hand, Shepard found in industrial laboratories that in general the more contact 
the better. Even in basic research groups within an industrial setting, he found more 
contact associated with higher performance. There is an apparent contradiction between 
the two sets of results; but I think it may be resolved by more knowledge of the nature 
of the task and the goals of the organization. One may hypothesize that in both settings 
some interaction with colleagues is better than none; but the optimum --the amount of 
contact needed for best performance — may be lower in individualistic than in a cooper- 
ative atmosphere. That concludes my remarks. 

G--I wonder if we could reach a hypothesis for possible future research, something 
to the effect that the greater the amount of contact (I don’t know how you measure it) the 
more prolific a group will be in the number of novel ideas they produce. How do you 
measure contact, by the way? What’s your operational definition for contact? 

S--We just asked them how often do you see these people --so many times a day, 

etc. 



Q--You don’t think that any further refinement is necessary? Seeing them is one 
thing, but seeing them for extensive discussions about problems at hand, etc. , is some- 
thing else. 
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S— I agree. In our measures we had to get something in a brief space, but more 
distinctions are needed. Shepard did a breakdown on types of contact -- work -oriented 
and sociability-oriented; he found that it was the work-oriented contacts that related 
mainly to group performance. 



A PROPOSAL FOR ESTABLISHING ULTIMATE CRITERIA 
FOR MEASURING CREATIVE OUTPUT 

J. H. McPherson 
The Dow Chemical Company 
Midland, Michigan 



S--Much of our discussion so far has indicated that we are more or less dissatis- 
fied with our methods of selecting our groups of "highly creative" and "less creative" 
people. Asking judges to select these groups based on the answers to such questions 
as "Who would you give research money to?" "Which men represent the kind you would 
like to hire more of?" "Which men have the best ideas?" have all seemed unsatisfactory. 
Most all of the researches in this field finish with the recommendation that considerably 
more time and attention be paid to obtaining an analytical evaluation of research perform- 
ance. 



How could this goal be achieved? At least three methods are available: (1) The 
trai n ing of the judges could be improved; (2) A tally could be made of the "creative 
products" produced by the scientist; and (3) A searching analysis could be made of all 
the creative products produced by the scientist and a summation made. 

This paper proposes that an examination of the products of research men is one 
of the best sources for ultimate criteria. What are some of the most usual creative 
products produced by a scientist? 



a. Patents 

b. Patent disclosures 

c. Publications 

d. Unpublished research reports 

e. Unprinted oral presentations 

f . Improved processes 



g. New instruments 

h. New analytical methods 

i . Ideas 

j . New products 

k. New compounds 



Using method 2, the tally method, we could merely add up the number of products 
in each of these categories, since many of them must go through some screening to ob- 
tain their classification, and make a judgment about the "creativity" of the scientist. 

One of the criteria used at G.E. to prove the worth of their Creative Engineering course 
is the number of patents produced by their graduates . 

Most research men would be reluctant to use number of patents alone as a suit- 
able criterion for measuring "creativity. " These would be some of their arguments: 

1 . There are many valuable creative acts that might meet some standard of 
creativity but are not patentable. (A new foaming-in -place polystyrene has 
been developed but may not be patentable because the patent lawyers be- 
lieve that prior patents pointed in this direction. ) 

2. There is a wide range of quality in patents, (One researcher may, because 
he feels that it is a "good thing” to get patents, develop one chemical 



compound after another and pile up quite an impressive list of patents. 

Another investigator, less socially motivated to seek patents, may seek 
one only after he has done work that meets his own high standards of sci- 
entific work. But, when they get ready to make quick judgments of people 
and they have available on their desk the number of patents and patent dis- 
closures that each scientist has they are inclined to make their judgments 
on this numerical statement, in spite of the realization that the quality of 
the patents should be considered. ) 

3. An investigator working in a fertile field where little work has been done 
may accumulate many patents while a worker in a well plowed field may 
find a patent hard to come by. 

4. Other investigators may be so motivated to pursue science for science’s 
sake that they neglect to patent a patentable product. 

5. Over a long span of time the frequency of patent issue is affected by social 
conditions and changes in patent laws. 

No doubt many other objections could be raised to the use of the number of patents 
as a criterion of creativity. If the patents were to be used, each patent would have to 
be subjected to a qualitative analysis. Each of the other listed products would have to 
be analyzed before it could be included. 

What methods could be used to make the analysis of each of these products more 
exact? The analyses will always be a matter of human judgment, the only gauge for 
measuring invention since objective criteria are lacking. We can make a start toward 
developing analytical methods for judging the quality of these products by reviewing 
some of the problems involved in determining the "inventivlevel" of patent applications . 
Since we wish to develop keener judges we might as well capitalize on the dilemmas 
faced by judges in the past. 

Section 3000 of Stringham’s patent law deals with "inventivlevel. " According to 
the law, one who originates novel technology has the right to obtain a patent only if the 
"quantum of novelty claimed rises to inventivlevel. " Inventivlevel is one of the law's 
standards, or norms or values. 

What are some of the characteristics used to determine "inventivlevel"? 

1. The first characteristic that is generally brought into the definition of inven- 
tivlevel is "qualified intellectual activity which carries the designation, creative strength, 
was used to solve the problem. " 

2. A second characteristic which is generally brought into the definition of in- 
vention is usefulness -- Fortschritt -- "Stride Forward". Here are some of the ar- 
guments involved in this judgment. 



"When one assumes that in every invention there should, in reality, be a certain 
Fortschritt, one must first of all define what Fortschritt means . . . What quantity of 
usefulness, then, must anything contrived include in order to obtain the designation 
invention? And in what direction shall this usefulness extend? And further, for what 
reason shall one call a new process for preparing photographic plates no invention 
when to be sure it represents no Fortschritt, no reduction in price, no acceleration, 
no greater sharpness in the pictures received, etc., but nevertheless presents an in- 
disputably new and original contrivance? 

"Finally, why should we not also call inventive that thought, which, with all its 
sagacity, signifies a step backward because of its injuriousness to culture? The legis- 
lative ukase can, certainly, declare expressis verbis the injurious and indecent inven- 
tion is not qualified for a patent. Not all that is useful is an invention and not every in- 
vention is useful. " 

3. A third characteristic that has been proposed is newness associated with the 
overcoming of special difficulties; every invention "offers something. Unusual, remark- 
able, and surprising. " An umbrella with a pen knife attached might be surprising but 
would not qualify. 

....... "When in a given case the chemical industry is not in* a position to expect 

a definite result, then we would have a case with something not logically derivable, the 
discovery of which would thus be an invention. " 

Here the matter depends not upon the result creating a surprise, but upon some- 
thing entirely different, namely upon the reasons which produce the feeling of astonish- 
ment; not every surprise permits us to conclude that we are in the presence of an in- 
vention. Reasons which cause surprise may lie in the most varied factors such as lack 
of correspondence between the external conditions and the content of the matter, unex- 
pectedness of the results, or unusualness of them. 

A German patent officer says, 

"They ask how I recognize an invention out of thi s mass of applications which 
seek after privileges. Very simple. While I look through one drawing after another 
I feel my attention riveted unexpectedly at times by one of them; from the details lying 
before me, I see immediately the spark of creative fire - -a nervous chill runs along 
my back; that, then, is an invention. " 

4. A fourth characteristic, "technical effect, " "economic effect, " "new effect, " 
"thoroughly new, " "new in principle, " is examined but rejected by Stringham. 

5 . The fact that a man with a previous record of technological achievement, who 
had the closest "old art" before him, actually did an appreciable amount of experiment- 
ing before achieving the novel instrumentality is always treated as relevant. 

6. Prior failure. The law is prone to find inventivlevel present where the alleged 
prior art never yielded commercial success. 



7. Prior skepticism. Success of a new instrumentation is particularly impres- 
sive if evidence shows that persons engaged in the branch of activity involved, had 
manifested a conviction that development like that of the new instrumentality offered 
no promise of success . 

8. Lag. When a judge becomes convinced that prior to the novel step contrib- 
uted by a patent, there had been an unfulfilled desire which the patent fulfilled, the 
patentee has pretty well won the inventivlevel branch of the case. 

"If the combination would have had practical value long before it appeared, if 
no impediment -- technical or commercial stood in the way, if during that time others 
had been at work upon the same subject, and if the new instrumentality was at once 
accepted as an answer to the old need, there is usually a just basis for the inference 
that inventivlevel has been achieved. 

"If when a need appeared it was fulfilled without extended delay by two or more 
independent workers, the absence of lag is evidence of non -inventivlevel, just as its 
presence is evidence of inventivlevel. 

9 If a patentee can prove a decided rise in business, following the introduction 
of the patented instrumentality he has greatly strengthened his position on the issue of 
inventivlevel - - if the business rise is: not due to economic conditions; not due to psy- 
chological change in fashion; not due to breakdown of purchaser resistance. 

10. Those who take licenses under the patent as well as by those who operate 
with the patented embodiment without license add to the confidence in establishing in- 
ventivlevel. 

11 . The instrumentality should possess novelty. There are certain kinds of 
novelty that are not associated with inventivlevel: compactness; convenienience; change 
of energy source; perfection of workmanship; purity of product. 

There are many considerations that must be applied to determine the: "novelty" 
of a combination; "novelty" of a new application; spatial and kinematic novelty; "novelty" 
of a deletion of a useless part; "novelty" of a substitution. 

Now these eleven considerations do not nearly exhaust the deliberations used in 
appraising a patent application, but they do suggest that each of the listed products of 
creative work could be appraised from a similar point of view and after similar arduous 
analysis. 

Ideas were listed as one of the products of the scientists . It is possible that 
member A of a laboratory group has an idea that some other man B carried to com- 
pletion to obtain the product for patent. If the judgments are made on products alone, 

A would not get credit. But if the man’s group leader is motivated to make his ap- 
praisals of men in an objective fashion, he will enter in his log that it was A who had 
the idea. The evaluation of the idea may be obtained partially by what B is able to do 
with it plus what criteria are developed for evaluating ideas. Having ideas that others 
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patent may be A’s primary method of contribution. 

After Scientist Jones has been in a laboratory for 5 years and his supervisor, 
who has been taught to analyze, has kept a log on him and analyzed each of his products, 
a '’fairly decent” criteria should result. 

At the present time the Dow Chemical Company has made provisions to advance 
scientists along two lines — supervisory (chemist - project leader - group leader - 
assistant lab director - lab director) and along scientific lines (chemist - project 
leader - associate scientist - research scientist). 

According to the director of our Patent Department, when a decision is to be 
nade about whether a person is to be advanced up the scientific ladder, a careful 
study is given to the number and quality of his patents as criteria in making the judg- 
ment. The patent director sees a need for making these judgments more explicit. 

To establish methods for analyzing products, busy industrial research people 
would need to become involved to a high degree. It is unlikely that they will become 
so if the primary purpose of the task is to validate psychological tests. But if the 
solution of the problem provides additional advantages such as deciding which men 
should be moved up the scientific ladder, a fairly accurate appraisal of each man's 
work, fairer rewards, etc. , they can afford the work involved in defining criteria and 
keeping the logs, etc., necessary to provide an accurate record. 

Since we will be struggling with t’ ultimate criteria problem for some time, I 
would like to make two suggestions: 

1. The labor involved in determining analytical methods for looking at each of 
these products and in developing methods for putting these analyses together would be 
considerable. Some research people feel that their intuitive quick judgment would be 
valid most of the time and tend to view such labor as a waste of time. Others feel that 
the mere adding of products would be sufficient. So I propose that to test these judg- 
ments a procedure somewhat like the following be tried: 

a. Three sets of criteria are collected on the same people. 

(1) A group of scientists are rated in the usual fashion - -quick judgments on 
creative level of the subjects. 

(2) A numerical count of "products" is made and a ranking established. 

(3) An analytical scheme for each product is developed and applied to all the 
products of each man. Then all the subjects are ranked according to 
these analyses. 

b. The relationship between these three rankings is obtained. 

2. The National Science Foundation might invite the National Patent Foundation 
to help with the problem. 
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The Patent Foundation 

The George Washington University 

Washington 6, D. C. 

The publication explaining their task indicates that they consider "re-examination 
of the standard of invention with respect to theoretical merit of invention and the ad- 
vance made in the practical art" as an appropriate research topic. 

C— I'd like to comment that I think this is a wonderul idea. I found in your re- 
marks certain similarities with what I’ve been saying for the past three years in the 
NRC Fellowship Selection Techniques Committee. I’d like to say this much that I have 
a feeling, whatever your correlation is between your three criteria, for theoretical 
reasons it’s going to be easier to predict the third one than it is to predict the first or 
second which are the ones we always get. This is the "seat -of -the -pants” reaction 
which I have had because you've got a cleaner thing in the third; that is, you can break 
it down and say, "Well it’s a performance criterion. " The others involve human judg- 
ment a good deal more. 

Q- -Isn’t this almost always true of your criterion problems: the ones that you 
would ideally like to have are the nearest impossible to get? 

S--But it may be more possible to develop ultimate criteria for measuring the 
performance of industrial researchers than for developing ultimate criteria for meas- 
uring the combat performance of bombardiers, a problem that gave Flanagan consider- 
able difficulty. 

2 — How about the problem of the differences between establishments? Now, 

Dow Chemical might do it this way, and another chemical company might do it differ- 
ently. How about making comparability across companies, for example, in research 
chemists? 

S— Probably the list of "products" given on the first page of this report is com- 
prehensive. If methods for measuring the quality of each of these products could be 
determined then they might be applicable in any research setting. 

C--Yee, but when you get to somebody who is individually keeping these logs and 
analyzing individual products there is apt to be considerable range in methodology. 

C— Well, look at it this way. I look at it like a coding system for analyzing the 
TAT. Now, you can use a judgmental method, or you can develop very objective scor- 
ing criteria which people can learn to apply. I gather the latter is what you are looking 
for. 



<J--This is what I’m talking about. After Scientist Jones has been in the laboratory 
for five years and his supervisor who has been taught to analyze has kept a log on him 
you may have something. How are you going to get all the companies to keep these logs 
and train their supervisors to do this? 
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C--Well, it will be difficult and it will be a twenty-five year project! 

S--It is probably fortunate for us that there are many administrative practices 
that are based on the performance evaluation. We can expect that considerable ’’other” 
effort will be devoted to the task of developing ultimate criteria. We can benefit from 
this effort. 
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THE RELATION OF INTELLECTUAL FACTORS 
TO CREATIVE THINKING IN SCIENCE 

J. P. Guilford 

University of Southern California 



S--This conference is concerned with the nature of creative thinking as exhibited 
by the scientist and with the means by which potentially creative scientists can be distin- 
guished from other individuals . With the word ’’talent" in the title, the emphasis is placed 
upon what the creative scientist can do --in other words, upon his aptitudes or abilities. 

Of all the human abilities, and the functions and processes that they imply, those 
in the intellectual area are undoubtedly most likely to be related to successful creative 
thinking in science. We can thus eliminate from consideration for the time being the 
strictly perceptual and psychomotor abilities. 

It will be the purpose of this paper (i) to present a quick survey of the intellec- 
tual abilities in terms of factor categories, of which there appear to be about forty 
that are now known; (2) to point out the system into which those factors seem to fall; 

(3) to consider in a general way how these factors may play a role in creative scien- 
tific work; and (4) to present some evidence concerning the relation of certain factors 
and factor scores to criteria of achievement in science and mathematics. The increas- 
ingly important role of mathematics in the sciences requires that we give mathematical 
aptitudes due attention also . 



The Structure of Intellect 

Let us first distinguish between ’’intellect” and "intelligence. ” Anticipating what 
is to follow, "intellect” is defined as the collection of memory and thinking functions 
and processes, which also become known as factors. The intellectual factors include 
a large number of thinking factors and a small number of memory factors. "Intelligence" 
has never been uniquely or satisfactorily defined. It is a shifting collection of intellec- 
tual factors, depending upon the investigator, the tester, or the intelligence test. The 
term has usually referred to a very limited aspect of intellect, as will be shown. 

The system of the intellectual factors, which now seems to be taking shape, in- 
cludes five groups of thinking factors, as well as the group of memory factors mentioned 
above. One of the larger groups is called "discovery factors" (see Table 1). A second 
large group is called "production factors" (see Table 2). Three smaller groups are called 
"divergent-thinking factors, " "evaluation factors, "and "symbolic factors. " We shall con- 
sider each group in turn. * 

A discovery factor is an ability to develop information out of what is given by 
stimulation. The discovery factors might have been listed under the heading of "cog- 
nition, " but t his term is too broad. They might have been listed under the heading 

* About half of the 35 thinking factors listed have been known for some time and are in- 
cluded in French’s re view of aptitude factors. Most of the remaining ones have arisen 
from research on aptitudes of high -level personnel (Project 150-044, under Contract 
N6onr-23810withthe Office of Naval Research) under the direction of the writer . 
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"induction, " but this term is too narrow. A discovery factor is exhibited in a test 
that calls for becoming aware of a class, a relation, a system, and the like. Such cat- 
egories you will find along the left-hand margin of Table 1. 

The columns of Table 1 deal with the kind of material in which the discovery is 
made. It is figural (perceived form), structural (such as letter patterns), or concep- 
tual (verbal concepts). Thus there are three abilities having to do with seeing relations 
and three for seeing patterns or systems. There may be three of the other kinds when 
all are known; for seeing classes, problems, and implications. There is not time to 
consider each factor individually here. Each factor may become somewhat meaningful 
by reason of its position in the matrix and by reason of the definitive tests given by 
name just below the factor name. 

The production factors (see Table 2) form a similar matrix. The kind of thing 
produced provides seven categories, as seen in the left-hand margin. Examples of 
things produced are words, ideas, expressions, and orders. Again, for each kind of 
thing produced the material involved — figural, structural, and conceptual — provides 
potentially three different factors. In Table 2 we have a number of vacant cells, but 
it would be reasonable to expect that many of these will be filled, and our future pro- 
ject research will be aimed at testing the hypotheses that they do exist, as the system 
implies . 

The divergent -thinking factors form a small group whose common feature seems 
to be that in order to make a good score in a test the examinee must allow himself to go 
off in different directions. This may be because some searching is necessary or it 
may be because the examinee is set to allow himself considerable freedom in his direc- 
tion of thinking. The test may require the solution of a problem or it may allow consid- 
erable latitude in its goal. In this group wq find that all factors suggest creative think- 
ing, but it can be shown that creative thinking is probably not confined to them . 

The evaluation functions have the general purpose of determining whether the end 
result of thinking, or whether any step in thinking, is good, correct, reasonable, or 
suitable. There are, of course, many criteria of the value of products of thinking, de- 
pending upon the circumstances. Thus, we have more than one evaluative ability. 

The symbolic factors have to do with the possession of symbols and abilities to 
manipulate them. Verbal comprehension is the overwhelmingly dominant component 
Qf verbal -intelligence tests. A good vocabulary test would often serve as well as an 
entire battery of tests, most of which are verbal. Numerical facility is an ability to 
deal with numbers. The other two symbolic factors represent processes that would 
seem to be parallel to certain operations in mathematics and symbolic logic. 

Although the last three groups of thinking factors do not go far toward generating 
matrices such as those in Tables 1 and 2, there are many indications that when their 
pictures are completed, they, too, may fit into three columns - - figural, structural, 
and conceptual. 

The seven known memory factors do seem to conform to such a pattern (see 
Table 4). It is not likely that any of the memory factors have much to contribute directly 
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to creative thinking, but we should not forget that available information is needed for 
this activity, and a good fund of ready information depends upon memory. 



Sons Suggested General Relationships 

The most striking implication of the list of intellectual factors for creative 
thinking in science is that, in the great variety of abilities represented there, we surely 
have among them the essential ingredients for successful scientists. We are also much 
better prepared to understand why the customary intelligence test or scholastic -aptitude , 
test has been of so limited value in predicting achievement in the sciences above the ' 
elementary levels. If the criteria for scientific achievement were something other 
than grades in courses and particularly if they stressed creative performance, it is 
safe to predict that the showing of such tests would be even less good. The average 
test of this character is usually limited in its content to not more than ten of the in- \ 

tellectual factors, none of which is among the more creative, divergent -thinking group. ^ 
What is more significant, the total or composite score of such a test is either domin- 
ated by one or two factors or it submerges all the factors in a rigid pool with the more 
pertinent factors often weighted unfavorably for prediction purposes. 

The next steps indicated by these considerations is to determine the relationship 
of every promising factor to creative performance in science students and scientists. 

The criterion of creative performance is, of course, a crucial requirement in achiev- 
ing this goal. No solution to the criterion problem is offered here, and no prescription 
except to insist that the criterion itself be a valid one . In view of the very large number 
of factors and the great variety of activities in which men of different sciences engage, 
we shall need a number of different criteria. Scientific ability is itself a very complex 
and shifting thing, depending upon the circumstances. 



Role of the discovery factors 

The discovery factors would seem to have many relationships to the work of a 
scientist. At some points, at least, scientific discoveries must depend upon the scien- 
tist's discovery functions. The use of the term "discovery" in both connections is no 
mere coincidence. The scientist is continually looking for relationships. Seeing re- 
lations in his data of observation is still the main source of his hypotheses, in most 
sciences. The three education -of -relations factors are probably of different levels of 
importance in different sciences and in different kinds of research in the same science. 
The perceptual variety might be more important to the user of a microscope whereas 
the conceptual variety might be more important for the interpreter of statistical data. 

Seeing classes has surely been important for those who have developed the clas- 
sificatory systems in botany and zoology. Here the factor o f perceptual classification 
may be relatively more important. Classifying activities are by no means limited to 

^ As found by L. V. Jones and P. C. Davis, who have factor analyzed the Binet 
and Wechsler tests, respectively. 



the biological sciences, and everywhere there must be much dependence upon the factor 
of classification that deals with conceptual material. 

Seeing patterns or systems would seem to be an activity basic to the formu^ on 
of laws, everywhere in science. Here we have three factors of potential usefulness. 

The spatial -orientation factor has been shown to have some relevance in advanced math- 
ematics . It should be important wherever the observer must keep his space bearings 
in looking at complicated material. The range of activity related to education of patterns 
in structural material is not yet known. The factor of general reasoning can be predicted 
to be of importance wherever problems of an arithmetic -reasoning type ire involved. 

The ability to comprehend problems cast in arithmetical form seems to be in general 
demand in science courses, but this differs from one science to another. How far the 
factor generalizes beyond this type of problem we do not know, but probably its utility 
is quite general. 

Sensitivity to problems is an ability in which, in the writer's experience, there 
are gross differences among graduate students. Prolmbly the most popular description 
of those who have it to the highest degree is that they possess a "natural curiosity. " 

But it is more than curiosity. I believe that this ability can be developed in students, 
if special attention is given to it. Also, where scientists work in teams, especially, 
where one person is weak in this respect he is often able to borrow from another's 
excess supply of problems. This last remark can be generalized to some extent. In 
a team of scientists, other weaknesses in certain factors in some members can be 
covered by strengths in other members. When we know where these compensatory fea- 
tures will work, we can make use of such information in building teams of scientists . 

Seeing implications, either in perceived material or in material one is thinking 
about, seems to have a value in science. This one thing, at times, may make the dif- 
ference between the laboratory assistant who works under direction and the inspired 
scientific leader who directs others. A study contrasting two groups, one high and 
one low in this ability, would be of considerable interest. 



Role of the production factors 



The production factors would seem to have relatively less to offer in our compre- 
hending of scientific thinking than the discovery factors, but some of them are probably 
relevant. The act of naming would arise when new terminology is needed, but this oc- 
curs on relatively rare occasions . The factors of word fluency , associational fluency , 
and expressional fluency would seem to be useful only in the process of writing reports, 
an activity that is by no means crucial to scientific discovery. Most scientists indulge 
in this activity however. If there are to be specialists in report writing, tests of these 
fluency factors should help to single them out. 

The factor of ideation fluency is one that is justifiably called creative. Thus far, 
tests of it have been confined to verbalized ideas. It is suggested by the system in 
Table 2 that there should be a fluency in thinking up figural ideas and also a fluency in 
thinking up structural ideas. The possible relation of either of these to production of 
ideas in science is not clear. Whatever the field, however, and whatever the scientific 
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problem, idea men are recognized and often prove their value. It may be that the 
present tests of ideational fluency, which severely limit the working time, are iibt 
realistic in the scientific context. ,The scientist obviously has more time in which 
to generate ideas. But even longer time intervals have their limits, and it remains 
to be seen whether the tests can distinguish between the man with one idea per day 
and the man with a dozen ideas per day. 

The education of correlates means, in popular terminology, the completion of 
an analogy. Much productive thinking proceeds in the form of thinking by analogy. 

Given a certain relation between two things in one field or context, what do we need 
to form such a relation between two other things in another field or context? Examples 
are numerous in which creative outcomes arise from thinking by analogy. We may 
expect three factors in this category. Thus the principle of thinking by analogy should 
have many and varied applications. 

The two abilities having to do with production of changes may have some inci- 
dental relations to scientific activity. Visualization i s the process of bringing about 
transformations of one kind or another in observed objects, or objects thought of visu- 
ally. Its relation to geometric thinking has been demonstrated, so that wherever geo- 
metric thinking is important in science we may expect the factor to be involved. Re- 
definition means a kind of flexibility of concepts, or more specifically, of meanings or 
uses of objects. Improvising probably depends upon it. Being ready to interpret an 
object or an observation in some new manner may also depend upon it. Revisions of 
classifications, laws, and theories may be related to the redefinition factor. 

Role of the divergent -thinking factors 

Another kind of flexibility has been found in connection with problem solving. 
Adaptive flexibility, one of the divergent-thinking factors, is very necessary in solving 
problems that require steps or solutions very much out of the ordinary. Old habits of 
thinking prevent some individuals from taking unusual steps, where other individuals 
more readily break out of the rut and strike out in new directions. 

The second kind of flexibility in the divergent -thinking list is called spontaneous 
flexibility. This shows up when flexibility is not necessary. It may indicate a lack of 
a strong goal idea or it may indicate a flexibility of goal. It might even be a handicap 
to a scientific thinker who has this tendency to a high degree, except when he needs to 
indulge in flights of fancy, with great freedom from an immediate goal. 

A factor of originality has proved to be quite measurable, at least in certain areas. 
It is measurable in terms of the number of clever responses given, in terms of remote 
or tenuous associative responses, and in terms of the number of idiosyncratic word- 
association responses. Tests of this character are verbal in content. It remains to 
be seen whether there are other originality factors pertaining to non-verbal areas of 
activity. Such a finding would not be surprising, and yet it would seem that the set to 
be unconventional, which the known factor seems to imply, should operate quite broadly 
regardless of the kind of material of thought involved. There is, however, seme danger 
that in any test of originality we shall have a large component of specific variance, which 
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may attenuate measurement of the underlying dimension of an appreciable extent. At 
any rate, it appears that we have taken large steps toward identifying individuals from 
whom to expect a larger probability of novel ideas . 

The factor of elaboration was found in connection with a study of planning abilities. 
It is the ability, given the broad outlines of a plan or given a starting idea, to develop 
the details needed to complete the picture . This ability may also be important in the 
planning of research. It should enable the individual to overlook no significant detail 
that should be considered in the designing of an investigation. 

Role of the evaluative factors 

The evaluative factors pertain to self criticism. They come into the picture when 
scientific ideas are to be tested in thought rather than in overt trials in experiments. 

Of all the* evaluative factors listed, that of logical evaluation is most promising as an 
aid to the thinking scientist. Are his conclusions sound? Do they follow logically from 
his premises? hi the development of theorems and hypotheses, starting with postulates, 
sensitivity to logical necessity is most important. The factor is probably best defined 
as sensitivity to logical necessity. 

The evaluative factors have a more general relation to the productivity of the 
scientist. If they are overly applied, they may provide inhibitions that work against 
fluency of ideas and the toleration of novel ideas . This is likely to happen, not because 
the individual has too much ability of die evaluative kind, but because of an over -criti- 
cal attitude which catalyzes their application. 

Role of the symbolic factors 

Among the symbolic factors, the factor of verbal c imprehension is known to be 
an important asset during the learning of the concepts of a science. A vocabulary test 
is therefore sometimes highly predictive of science grades, in lower -division science 
courses particularly. Beyori that level, however, the factor appears to be of less and 
less importance. It probably has little or no predictive value in distinguishing creative 
from non -creative scientists. 

The factor of numerical facility has some predictive power wherever operations 
or thinking with numbers is involved in learning a science or in operating in it. Such 
places can be readily recognized and measures of this factor would be called for ac- 
cordingly. 

The factors of symbol substitution and symbol manipulation are so litde under- 
stood that we can make only tentative suggestions regarding their usefulness in science. 
It was suggested earlier that they may be useful in connection with the mathematician. 

By the same token, they should have some relation to the performance of the theoretical 
physicist, or any type of scientist who does considerable thinking by assigning symbols 
and manipulating them to develop theorems and hypotheses. This possible relation 
should be investigated. 
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Some Early Indications on Factor Validities 

I # ■ , 

The relation of any factor to a criterion of success in some activity such as 
scientific performance can be determined in two ways. A factor analysis can be done, 
including the criterion along with the test variables analyzed. This yields an. estimate 
of the correlation of the factor with the criterion, and it is meaningful information. 

The other approach is the ordinary ere, correlating the scores from a good test of the 
factor with the criterion. This yields the practical information of how much prediction 
we can expect in practice from the measures that we can obtain. It is less informative 
with regard to the importance of the factor, particularly if the test is substantially 
loaded with other factor components, as is so often the case. 

A word of warning should be expressed in this connection. The correlation co- 
efficients between single -factor tests and a criterion are usually lower than those be- 
tween factorially complex scores and the same criterion, for example, scores from a 
composite intelligence test. This leads some investigators to conclude that scores 
from factor tests are less predictive and therefore less useful than scores from stand- 
ard, composite tests. This conclusion is unfair and ignores the fact that several fac- 
tor scores combined with appropriate weights will almost surely yield greater accuracy 
of prediction than an ordinary composite score that covers the same factors and only 
the same factors. The comparison should be based upon composite scores from both 
kinds of source. 

In the validation studies reported in the remainder of this paper, course grades 
are used extensively as criteria. This is not done with any belief that they reflect 
creative performance, but largely because they were available and it was expected 
that they would throw some light on the .generality of the factors in diverse areas of 
intellectual endeavor. In the over -all problem of selecting successful scientists, it 
can be argued that these criteria are not unimportant, because, in our present edu- 
cational system, students must do well in undergraduate courses in order to be accep- 
ted for graduate work and thus have the opportunity to become scientists. 

As an example of validities of factor scores. Table 5 presents correlations of 
five such scores with average grades in certain lower -division science and mathe- 
matics courses at the University of Washington. The factors represented are verbal 
comprehension (V), general reasoning ( GR), numerical fac ility (N), spatial orientation 
(SO), and visualization (Vz). Greatest validity is usually found for factors V and GR, 
as we should expect at introductory levels of these subjects. Factor N plays its great- 
est role, in this list of subjects, in mathematics, and next in some of the sciences 
where number work is more common. Factor SO plays little role in the evaluations 
of students in these six courses. Factor Vz bears some small relations to the biologi- 
cal-sciences grades, perhaps because of the microscopic work involved and because 
of the component subjects of anatomy and physiology. 

A study of multiple predictions of grades was not made, but because these factor 
tests have generally low intercorrelations we should expect substantial multiple corre- 
lations. 
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In the next study of ours that I shall mention, about ten of the intellectual factors 
were involved, including some of the more directly creative abilities . The criteria 
were grades in scientific and mathematical courses in the freshman year at the U. S. 
Coast Guard Academy . A factor analysis was done, including course grades along 
with test scores . 

Table 6 shows, first, the correlations of any consequence between each of three 
factors and one or more courses. The factor of visualization correlates more than 
. 20 with Engineering Drawing and Descriptive Geometry (we combined engineering 
drawing and descriptive geometry together as one variable because the intercorrela- 
tion of the two was very high), Celestial Navigation, and Physics. The reasoning-corn - 
posite fa ctor (general reasoning with some variance in education of patterns and per- 
haps other factors) correlates substantially with grades in Algebra and Trigonometry. 
Adaptive flexibility correlates to a small extent with the same course. 

Next, correlations between certain tests of the factors and course grades are 
given in Table 6. The creative factors of seeing problems, associational fluency, 
ideational fluency, and originality have no significant correlations with grades in science 
and mathematics courses, hence are not mentioned in the table . We should probably 
e^qject no correlation, since in freshman courses, the emphasis is placed on mastering 
the fundamentals of the subjects, not in producing new ideas in them. The lone signif- 
icant correlation between a test of expres signal fluency and Astronomy invites specu- 
lation. The hypothesis of a chance correlation is rejected at the . 01 level; hence, 
some alternative hypothesis is called for. The correlation might mean that the in- 
structor in that course gave essay-type examinations and his marking reflected his 
recognition of good writing. 

Looking down the list of factors in Table 6, we see that the relatively new factor 
of education of correlates has some promise of predictive ability in connection with 
two courses - -Celestial Navigation and Astronomy. Adaptive flexibility scores have 
small but significant correlations with three courses — Celestial Navigation, Engineer- 
ing Drawing and Descriptive Geometry, and Physics. Neither of these factors has pre- 
viously been explicitly considered in connection with predicting achievement in science 
and mathematics; hence, they promise to add something to prediction. 

In a second Coast Guard class a similar validation study was made, but without 
a factor analysis of the criterion variables. The experimental tests in this study were 
concentrated on the factor of general reasoning. An improved test of this factor has 
significant correlations vath all three mathematics courses and all three scientific 
courses, as shown in Table 7 . One striking fact in this table is the poor showing of 
a test of verbal comprehension. This is some indication of how poorly a typical verbal - 
intelligence test would have done in this situation. Only the score for the factor of 
education of patterns had no significant correlations. The score for logical evaluation, 
a factor rarely or never covered in aptitude testing, had three significant coefficients.* 

Table 8 shows what could be done by way of improving predictions of grades by 
combining scores. Of special interest here is an entrance test of mathematical achieve- 
ment, which should be the best known predictor for grades in the mathematics courses, 
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and possibly in some science courses like physics. In four courses out of five, a 
composite of factor -score tests gives better prediction than the mathematics -achieve- 
ment test provides. In every case, a composite of this test plus factor tests improves 
prediction. In either respect, the value of the factor tests is demonstrated for this 
sample. 

A recent study on the prediction of mathematics achievement, by John R. Hills, 
may be cited as a model for validation studies, in many respects. Space does not 
permit giving many details. First, as a preparation for the study, he selected 16 
factors that he thought might be related to success in mathematics at higher levels 
of instruction. He asked a number of teachers of mathematics in a number of univer- 
sities to rank -order the defined factors for importance to the success of the advanced 
mathematics student. There was very high agreement among the rankings given by 
different mathematicians. Among the highest -ranking factors were education of pat- 
terns, originality, adaptive flexibility, and logical evaluation . Among the lowest were 
verbal comprehension, numerical facility, and sensitivity to problems . 

Depending partly upon this information, Hills selected nine factors for investiga- 
tion. Although they said numerical facility was not important, he doubted that, and 
put numerical facility in as one of the factors to be validated. The best available test 
for each factor was administered to students majoring in mathematics, physics, and 
engineering, distributed in three institutions of higher learning in the Los Angeles 
area. The criteria of mathematical performance that he used were of three kinds. 

One was an average grade in mathematics courses. A second was a score in a pro- 
ficiency examination in advanced mathematics taken by engineering students. A third 
was in the form of ratings. Students in physics and engineering were rated by their 
instructors on how well they could use mathematics in their fields . Students in mathe- 
matics were rated on the probability of their completing satisfactorily a program of 
graduate study, earning the Ph.D. 

A highly summarized tabulation of the results is given in Table 9. There were 
23 groups of students in which correlations with die nine tests were estimated. There 
is some overlapping of groups, so the estimates are not all experimentally independent. 
Curricula, criteria, and institutions and some of their combinations vary from one 
group to another. The summary thus takes many liberties, but may nevertheless give 
us some rough ideas and suggest hypotheses. 

The factors of verbal comprehension and general reasoning, which we find to 
have some validity in lower -division mathematics courses in other studies, seem to 
have no relation to achievement at higher levels. The number factor shows low but 
consistently positive relations to achievement for engineering students. It appears 
to have no validity for physics students and its validity for mathematics students could 
not be determined. 

The test of spatial orientation tends to be significantly related to success in math- 
ematics. for engineering students and to ratings of mathematics students, but tends to 
be negatively but insignificantly related for physics students. Visualization is related 



to mathematics achievement in some circumstances but not in others, with a very 
strong preponderance of positive correlations. 

Adaptive flexibility proved to have the highest median correlation but one of the 
greatest ranges, and quite a number of significant differences between estimates . 

There is a trend toward high correlations for physics students. For engineering stu- 
dents there is a trend of moderate validities at one institution and for low (insignificant), 
negative correlations at another institution. Logical evaluation has a strong prepon- 
derance of positive correlations, with the significant coefficients at two of the institu- 
tions only. 

The generally zero validity for the factor of education of patterns, whereas the 
consensus of mathematicians rated it highest in importance, calls for some comment. 
The factor was described to them as ’’the ability to discover a principle, rule, or 
system. ’’ It was later discovered that the test for this factor (Circle Reasoning) actu- 
ally measures a factor that belongs in the structural column in the system of intellec- 
tual factors (see Table 1). It may be that the ability that the mathematicians had in 
mind pertains to conceptual material, hence the Circle Reasoning test entirely misses 
it. 



The results for originality are interesting. Although this trait was ranked next 
to the top of the 16 by mathematicians, more than half of the correlations with criteria 
are negative a nd the median r is close to zero. One could hypothesize from this that 
a docile, conventional, grade -getting attitude that pleases the instructor still commonly 
prevails and is rewarded at upper -division and graduate levels . At one institution, 
however, where originality ranked highest, in a small class of graduate mathematics 
students, the Olmstead-Tukey corner test shows a positive correlation with grades in 
graduate courses significant at the .05 level. 

One of the most notable results from the Hills study is the great variability of 
validity coefficients, varying as they do from institution to institution, curriculum to 
curriculum, and criterion to criterion. A large part of this can no doubt be attrituted 
to differences in policies and practices in teaching, examining, and marking as well 
as to differences in demands of different mathematics courses upon kinds of talent. 

This result emphasizes, again, the great need for careful studies of criteria. Most 
important of all is the question: are instructors emphasizing the qualities that they 
think they are and the ones that they believe to be important in the subjects they teach? 

If originality is the most important quality to be stressed in teaching and in evaluation 
of student performance, is this quality given sufficient opportunity and sufficient atten- 
tion in teaching and in marking students? Perhaps here, after all, lies an important 
key to the production of creative scientists. 

The last two small studies that I shall mention are concerned with personnel 
who are mostly beyond the educational ladder. One group is of aircraft engineers 
who are chiefly concerned with problems of designing aircraft and the other is of 
operations analysts, who undertake problems of great complexity. Each group had 
been administered certain factor tests . 
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In the engineering group, the criterion was completely objective so far as the 
investigators were concerned. It was the rate of increase of pay grade, which is, 
of course, dependent upon many considerations. In view of the work the engineers 
had been assigned to do, creative output would presumably carry much weight. In 
samples from 52 to 65 in number, the significant correlations are with certain tests, 
as shown in Table 10. I apologize here for not reporting all the tests which had insig- 
nificant correlations, too, but show only where factors possibly have some positive 
thing to offer. Except for the factor of visualization, the factors significantly related 
are relatively new. Thus, we have promise of useful additional predictions beyond 
those that may be available from the use of tests previously available. 

The criterion for the operations -analysis group was in the form of an over -all 
rating of effectiveness in performing the duties of the job. The operations analyst 
presumably encounters mostly novel problems. In Table 11 we see that all of the 
validity coefficients are positive, four of them being significant and two others near 
the borderline of significance. The fact that this criterion seems to be predictable 
from tests of disparate factors whose intercorrelations are generally low, promises 
a very substantial degree of multiple prediction, if such results hold up in much larger 
samples . 



Summary and Conclusions 

In this paper I have presented a very brief look at the developing list of intellec- 
tual factors and the kind of system in which they seem to fall. The thinking abilities 
fall in five groups: discovery factors, production factors, divergent -thinking factors, 
evaluation factors, and symbolic factors. A group of memory factors completes the 
list. 



In relation to creative scientific activity, it would seem that the discovery fac- 
tors and divergent -thinking factors have most to offer. Factors from the other groups 
may play their roles in scientific work. The roles of all the factors and their rela- 
tive degrees of importance are still to be determined. 

Some empirical evidence was offered to indicate that many of the new factors 
bear significant relationships to criteria of success in the steps of learning sciences 
and mathematics and in practical scientific and technological work. 

In the way of general conclusions, two principles seem to be indicated: 

1. Scientific aptitude, including ability to do creative work in science, is a 
many-sided phenomenon, involving many different abilities, depending upon the science 
and the particular steps within the science. 

2. Most of the differences in what scientists are able to do can be accounted 
for in terms of the intellectual factors, some of which contribute more directly and 
some less directly to successful creative performance. 

These principles lead to several implications: 
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1 . Scientific performance, including the creative aspect, should be assessed 
in terms of appropriate criteria, which should reflect all the significant abilities, 
properly weighted. 

2. Systematic and comprehensive validation studies should be made, based upon 
die factor concepts, utilizing factor tests and appropriate criteria. 

3. Meanwhile, efforts should be devoted to deter ming whether still undiscovered 
abilities exist, as the system indicates. If found, their possible relations to creative 
scientific performance should also be investigated. 

4. A still different line of investigation should be aimed at determining what 
educational procedures are optimal for developing creative scientists. In this con- 
nection the factor concepts should also be useful tools. This line of research would 
involve two approaches: 

a. Examination of present educational policies and practices to determine 
whether any of them work seriously against such development. 

b. Experiments trying out procedures that would seem favorable for such 
development. 

Q--Did I understand you correctly in saying that you regarded the memory fac- 
tor as not related to creative work? 

S— Not directly, no. Of course, information is needed for creative work. If 
one has the information in his head, rather than having to search in a library, it's 
all to the good, isn’t it? 

C— Well, it's got to be there though before he can use it. 

S--Yes, either he has to recall it or get it out of a book or from new experience. 

C--Well, possibly what you mean is that he has to have it, but he doesn't neces- 
sarily have to have been able to learn it fast. Maybe, the memory factor measures 
the extent to which he can rapidly acquire information. 

S— Is it a matter of speed of learning or goodness of retention? Is that your 
question? 

<C--But I mean if he plugged away at it very slowly, if he had the information -- 
he's got it, and can use it even though he may not be very good at the memory factor 
as such. 

(J- -Doesn’t his memory filing system make a difference? The way in which he 
has the material organized in his mind? 

S--That affects his usability of it, yes. 



C--I have a general question or a general comment, rather, on the method of 
procedure here. I'm wondering about the factors such as the evaluative group factors 
on the one hand, and on the other hand originality or the divergent thinking factors . 

You can reason that a good scientist has to be both original and also a good evaluator. 
Now, maybe if the person is a bad evaluator it is better if he is not so original. 

5>- -Well, somebody else can correct him if he doesn't correct himself. It's 
better, of course, if he'll do it himself. 

Q--Well, I'm just making a hypothesis here. If he's a poor evaluator, he might 
get better grades if he isn't so original and he doesn't go off on wild tangents. On the 
other hand, if he is a good evaluator, maybe it’s better if he is original. In other 
words, maybe one of the reasons why you don’t get more of a correlation between origi- 
nality and grades in mathematics is because this evaluation factor is floating around 
there. Sometimes, it is good and sometimes it is bad. Unless, you hold that constant 
so to speak, you won't really be able to utilize the full predicted power of the original- 
ity. What you may need to do here is look at these two variables simultaneously or 
look at the possible combinations or the configuration or the pattern, I was wondering 
if you had thought of doing that kind of configuration analysis or pattern analysis or 
rather than just one -by -one kind of analysis? 

S— Does your statement imply a curvilinear relationship between the factors? 

(>-Nq, not necessarily. The factors themselves might be unrelated. I mean 
whether a man was high in originality might be unrelated to whether he was high or low 
in evaluation. What I mean is a kind of interaction affect. The two independent vari- 
ables may be unrelated, and they still can interact with each other in predicting the 
criterion in a pattern kind of way. 

S--The answer to your question is no. We have not considered the pattern ap- 
proach. 

C— I was just thinking about that when looking at your originality factor and won- 
dered why it didn’t come out stronger as to whether it might be confounded by some- 
thing like evaluation. I know that two of us were talking yesterday about the application 
of some pattern techniques for predictive purposes. 

“Have any of the factors that you report here or have worked with in the past 
had curvilinear relationships with a criterion? 

S --Apparently not. Of course, the correlations are pretty low, and it is then 
hard to determine what the regression is like. I don't think so. This ought to be in- 
vestigated. We would need a larger range for such an investigation, I think, too. 

C--One of the basic assumptions underlying the factor approach seems to be that 
there are generalizable abilities in skills or traits . The kinds of studies that we have 
been doing seem to suggest that a person who is a very original physicist may be a 
very dull and almost stupid social scientist or humanist, and we have the problem then 
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of whether some of these factors may be real only in the specific field, partly because 
of the kind of training we have. We, for example, may teach individuals to develop 
principles or apply principles in a particular subject field, but we do not help him to 
transfer this particular skill to another field. I think one of the basic hypotheses which 
could be put to experimental test is whether some of these factors in their generaliz- 
able.form can predict and distinguish between creative and non -creative individuals, 
as contrasted with some of these factors more nearly present within or close to the 
subject matter under consideration. 

S--Some of the factors we are talking about may be rather limited in scope, and 
you couldn't predict much from such factors. 

Q - -Was the previous comment that you think as I tend to think too, that factors 
may be specific to a given population? 

C — No, I'm trying to say that the educational system we now have in which an 
individual may learn science quite unrelated to mathematics, quite unrelated to 
social sciences, quite unrelated to literature, may be such that individuals develop 
highly specific skills . Some of our students at our university become very original 
when working on physics problems but almost are dull and stupid when working on 
social science problems and the like. So in originality, some of these discovery 
factors, etc. , may be present, but only within the context of a particular field mate- 
rial. 



S- -Are you assuming equal knowledge in both fields? 

C-Yes. 

C--Well, that’s about the same thing that I'm saying. This thing of populations 
framed in a certain way will make a difference in the fact that the patterns that come 
out for that group may be different from those that are trained in another way. 

C--I'm also suggesting that it may be necessary to test in relation to a particu- 
lar subject field rather than in more general sorts of things. As, for example, in 
the development of Plot Titles test. We have found this test not to work at all for 
some of our creative scientists. But we're still assured that they are original and 
can cook up one idea after the other. But they don’t seem to be able to do it in this 
general form. 

Q- -I’d like to raise another question on this originality. On the originality tests 
what you try to do in a short period of time is to see how many original products a 
person can produce in ten minutes, fifteen minutes, or something of that sort. Now, 
it strikes me that when we are talking about originality in mature research, maybe 
the person that comes up with one idea a year has a tremendous idea. The short 
sample kind of thing in originality tests may not measure the kind of long-term origin- 
ality. 

S- -That's still to be determined --we don't know. 



82 







C--From what we do know about productive people, it is clear that voluminous 
productivity is the rule rather than the exception for persons who do produce some 
idea which is clearly original and valid. If you produce one , you're ver3f likely to 
produce a lot. 

Q- -Doesn't this production tend to be, though, in a certain subject or area? 

Or if you usually do suggest that this generalized productivity is in a whole flock if 
areas, you think of Leonardo da Vinci or somebody like that perhaps -- isn't that 
pretty rare? 

S--The question is whether da Vinci was, after all, an isolated case or whether 
he was a pattern. My present prediction is that there is a generalizable trait of orig- 
inality and it will apply in different fields. 

C- -Including in social behavior. Some of the evidence we have on the correlates 
of originality, using a composite of scores, supports this. 

S--Now, if originality does not apply generally, it might be because other fac- 
tors are responsible. 

C--It may be blocked or facilitated, then, in one or another field. 

1 •» 

S--It might be because of a lack of knowledge. 

<C--I guess motivation would also make quite a bit of difference. 

Q--In connection with an earlier question, I was wondering about the comment 
of assuming equal knowledge here; I v/as wondering if you really did assume equal 
knowledge. This is the same question that Dr. Guilford raised, and it seemed to me 
to be crucial here. I wouldn't assume it as he described the situation. If you had 
trained someone to be productive in physics, I don't think that you could assume from 
any people that they would have acquired equal knowledge in the social sciences. 

C--I was thinking of our general education program where all the students take 
the same subject matter and the same examinations so that we can be pretty sure that 
they have all had at least common exposure or equal exposure to these materials . 

And yet we find that a student who seems to be unusually skilled in applying principles 
in the social sciences may have great difficulty in, although he may do equally well 
in the informational sections of, for example, the physical sciences. He may not do 
as well in applying principles in the physical sciences or humanities, as the case may 
be. My feeling is that it is quite possible that if we taught some of these skills so 
that the students would see the relationship between applying principles in the social 
sciences and in applying them in the physical sciences and the other fields, that some 
of the generalized skills might work quite well. But where subject matter is taught in 
the separate compartments of our curriculum at the moment and a student is not taught 
to relate an original idea here to an original idea over there, I don't think the general- 
ized test will work . 

b 

t 

X 
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C--I have a comment like some of those made before on pattern analysis of the 
configural scoring vs. looking along some linear single scale. As 1 recall, in one 
study, the researcher first divided the total group of undergraduate students and did 
an inverse factor analysis on personality measures and came out with a social rela- 
tionship factor. Studying separately those above the median on this factor vs. those 
below - -on those who have very good social relationships -- the I. Q. correlated prac- 
tically zero with the criterion. And on those with very poor social relationships the 
correlation between I.Q. and the criterion was something in the 80’ s. 



Table 1. Factors in thinking — Discovery factors 8 



Type of thing 
discovered 


Type of Content 


Figural 


Structural 

i 


Conceptual 


Relations 


Education of per cep- 
tual relations 

(Figure Analogies) 
(Figure Matrix) 


Education of struc- 
tural relations 

(Seeing Trends II) 
(Correlate 
Completion II) 


Education of concep- 
tual relations 

(Verbal Analogies I) 
(Word Matrix) 


Classes 


Perceptual classification 




Verbal classification 


(Figure Classification) 
(Picture Classification) 


(Word Classification) 

(Verbal 

Classification) 


Patterns or 
Systems 


Spatial orientation 

(Spatial Orientation) 
(Flags, Figures, Cards] 


Education of patterns 

(Circle Reasoning) 
(Letter Triangle) 


General reasoning 

(Arithmetic Reasoning) 
(Ship Destination) 


Problems 






Sensitivity to 
problems 

(Seeing Problems) 
(Seeing Deficiencies) 


Implications 


Perceptual foresight 

(Competitive Planning) 
(Route Planning) 

I 

i 




Conceptual foresight 

(Pertinent Questions) 
(Alternate Methods) 

Penetration 

(Social Institutions) 
(Similarities) 



a Factor names are underlined. Representative tests are given by title in paren- 
theses . 



85 



Table 2. Factors in thinking -- Production factors 



Type of Product 


Type of Content 


Figural 


Structural 


Conceptual 


Names 


Object naming 

(Color Naming) 
(Form Naming) 




Naming Abstractions 

(Picture -Group Naming) 
(Word -Group Naming) 


Words 




Word fluency 

(Prefixes) 
(First Letters) 


Associational fluency 

(Controlled 

Associations II) 
(Associations III) 


Ideas 






Ideational fluency 

(Plot Titles) 
(Consequences) 


Expressions 






Expressional fluency 

(V ocabulary Completion) 
(Similes) 


Correlates 


Education of correlates 

(Correlate Completion) 

(Figure Analogies Completion) 




Order 






. Ordering 

(Picture Arrangement) 
(Sentence Order) 


Changes 


Visualization 

(Spatial Visualization) 
(Punched Holes) 




Redefinition 

(Gestalt Transformation) 
(Object Synthesis) 
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Table 3. Other factors in thinking 



Factors Representative tests 

Divergent -thinking factors: 

Adaptive flexibility (Match Problems) (Insight Puzzles) 

Spontaneous flexibility (Brick Uses) (Unusual Uses) 

Originality (Plot Titles, cleverness) (Symbol Production) 

Elaboration (Planning Elaboration) (Figure Production) 

Evaluation factors: 

Perceptual evaluation (Probably not a unity) 

Logical evaluation (Logical Reasoning) (Inferences) 

Experiential evaluation (Unusual Details) 

Judgment (Practical Judgment) (Practical Estimations) 

Speed of judgment (Color -form Sorting) (Social Judgments) 

Symbolic factors: 

Verbal comprehension (Vocabulary) 

Numerical facility (Numerical Operations) 

Symbol substitution (Sign Changes) (Form Reasoning) 

Symbol manipulation (Symbol Manipulation) (Sign Changes II) 



Table 4. A matrix of memory factors 



Thing or Aspect 
involved 


Type of Content 


Figural 


Structural 

1 


Conceptual 


Associative 

connections 




Rote memory 

(Word-Number) 
(Color -Word) 


Meaningful memory 

(Sentence Completion) 
(Related Words) 


Substance 


Visual memory 

(Reproduction of 
Designs) 

(Map Memory) 

Auditory memory 

(Musical Memory) 
(Rhythm) 


• 


Content memory 

(Memory for Ideals) 
(Limericks) 


Span 




Memory span 

(Letter Span) 
(Digit Span) 


Integration I 

(Signal Interpretation) 
(Combat Planes) 
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Table 5. Validity coefficients of factor -test scores from Farts of the Guilford - 
Zimmerman Aptitude Survey for predicting grades in certain science 
courses at the University of Washington a 



Part: 

Course 

Factor: 


I 

V 


II 

GR 


III 

N 


V 

SO 


VI 

Vz 


. N 


Biology 


.61 


.45 


«*H 

O 

• 

1 


.19 


.26 


91 


Botany 


.45 


.35 


.27 


.13 


.22 


155 


Chemistry 


.32 


.38 


.27 


.13 


.18 


707 


Mathematics 


.29 


.37 


.35 


.12 


.18 


894 


Physics 


.34 


.29 


.17 


.11 


.13 


520 


Zoology 


.34 


.39 


.22 


.08 


.21 


357 



a Courtesy of E. E. Dudek, in a personal communication. 



Table 6. Relations of factors and factor scores to grades in freshman science 
and mathematics courses at the U.S. Coast Guard Academy (class of 



1957). (N= 110) 

Factor 

Factor Course Loading 



Visualization Engineering Drawing and Descriptive Geometry ... .56 

Celestial Navigation 24 

Physics I and II 25 

Reasoning composite Algebra and Trigonometry .40 



Adaptive flexibility Algebra and Trigonometry 22 



Factor score for 



Course 



Correlation a 



Expressional fluency Astronomy . 25** 

Education of correlates Celestial Navigation 21 

Astronomy , 20 



Adaptive flexibility Celestial Navigation 22 

. Engineering Drawing and Descriptive Geometry ... .22 

Physics I and II 23 



Visualization 



Engineering Drawing and Descriptive Geometry . . . 




Verbal composite 

(Probably includes ed- 
ucation of conceptual 
relations as well as V) 



Algebra and Trigonometry 

Analytical Geometry and Calculus 

Celestial Navigation 

Astronomy 

Physics I and II 



.19 

.23 



.22 

.21 

.21 



Reasoning composite 

(Probably includes ed- 
ucation of patterns as 
well as GR) 



Algebra and Trigonometry 

Celestial Navigation 

Astronomy 

Physics I and II 




.24 

.22 

.27 



a All coefficients reported here are significant beyond the .05 level. 

Jj 

Significant beyond the . 01 level. 
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Table 7 . Relations of factor scores to grades in six science and mathematics 
courses at the U. S. Coast Guard Academy (class of 1958). (N = 116) 



Factor score for 



Course 



Correlation a 



Verbal comprehension Nautical Astronomy 20 

Numerical facility Algebra and Trigonometry 21 

Visualization Analytical peometry and Calculus 22 

Descriptive Geometry 29^ 

Nautical Astronomy 27^ 

Physics 25k 

General reasoning Algebra and Trigonometry 25 b 

Analytical Geometry and Calculus 39b 

Celestial Navigation 18 

Descriptive Geometry 26^ 

Nautical Astronomy 35b 

Physics 4(P 

Logical evaluation Analytical Geometry and Calculus 22 

Nautical Astronomy 29^ 

Physics 25 b 



a All coefficients reported here are significant beyond the .05 level. 
^Significant beyond the . 01 level. 









Table 8. Some multiple correlations of factor-score composites with certain 
science and mathematics courses at the U.S. Coast Guard Academy 
(class of 1958). (N = 116) 



Course 


Factors 

involved 


Corr. with 
(prior) Mathe- 
matics Achiev. a 


Multiple R 
without 
Math. Achiev. 


Multiple R 
including 
Math. Achiev 


Algebra and 


HCP 0 


.40 


. 29 (. 25p 


.44 (.41) 


Trigonometry 


N 










Vz 









Analytical Geometry 
and Calculus 


HCP 

LE 

TEM d 

GR 


.32 


.44 


(.40) 


.49 


(.46) 


Descriptive 


HCP 


.18 


.39 


(• 36) 


.40 


(.36) 


Geometry 


GR 














Vz 












Nautical Astronomy 


LE 


.19 


.43 


(.39) 


. 44 


(-39) 



GR 

Vz 

V 



Physics I and II LE .34 . 41 (. 38) . 48 (.45) 

GR 

Vz 



a An entrance examination on preparatory mathematics. 

in parentheses is the "shrunken" correlation. 

c HCP stands for a factor called "handling complex procedures . " It may prove to 
be the same as Integration I, a memory factor. 

j 

°TEM stands for a factor called "trial -and-error manipulation. " It may prove 
to be the same as symbol manipulation. 



92 






Table 9. Summary of validity coefficients for nine factor scores for predicting achieve- 
ment in advanced mathematics a 



Factor 


Range of 
correlations ® 


Median 

r 


Number of 
positive 
r’sc 


Number 
of significant 
r's d • 


Verbal comprehension 


-. 37 to +.28 


+.06 


14 (of 19) 


0 


General reasoning 


-. 44 to +. 31 


+.03 


13 


1 


Numerical facility 


+. 01 to +. 44 


+.24 


19 (of 19) 


4 


Spatial orientation 


- . 31 to +. 68 


+. 26 


17 


10 


Visualization 


-.25 to +.44 


+; 28 


20 


3 


Adaptive flexibility 


-.31 to +.68 


+.33 


17 


6 


Logical evaluation 


-. 29 to +.72 


+.19 


20 


4 


Education of patterns 


-.31 to +.38 


+.11 


15 


1 


Originality 


-. 46 to +.21 


-.02 


10 


0 




3 . 

Based upon data from J. R. Hills, 
b 

From 23 somewhat overlapping groups, from three different institutions, three 
different curricula, three different criteria, and with degrees of freedom vary' 
ing from 9 to 73. 

From a total of 23 estimates unless otherwise stated. 

^Beyond the . 05 level . 
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Table 10. Validity; of factor tests for predicting pay -rate increments of aircraft 
engineers (N varied 52-65). a 



Test 


Principal factor 


Correlation j 


Symbol Manipulation 


Symbol manipulation 


. 35 c ] 


Circle Reasoning 


Education of patterns 


3lb 1 

.31 j 


Spatial Visualization 


Spatial visualization 


. 3lb i 

j 


Sentence Analysis 


Ideational fluency 


• 

co 

o 


Correlate Completion 


Education of correlates 


. 36° | 

j 


Match Problems 


Adaptive flexibility 


J 

■ 31 b j 


a Courtesy of Lockheted Aircraft Corporation. 


] 

i 


^Significant beyond the 


. 05 level. 




Significant beyond the 


.01 level. 


X 



Table 11„ Validity of factor tests for predicting ratings of over-all job proficiency 
of operations analysts in an aircraft manufacturing organization (N varied 
19-20) . a 



Test 


Principal factor 


Rank-difference 

correlation 


Ship Destination 


General reasoning 


.55 b 


Competitive Planning 


Perceptual foresight 


.42 


Spatial Visualization 


Spatial visualization 


.46 b 


Correlate Completion II 


Education of correlates 


.29 


Figure Analogies 


Education of perceptual relations 


.32 


Number Series 


(A complex of factors) 


.50 b 


Logical Reasoning 


Logical evaluation 


,47 b 


Vocabulary 


Verbal comprehension 


.41 


Numerical Facility 


Numerical facility 


o37 



a Courtesy of Lockheed Aircraft Corporation., 
^significant beyond the . 05 level. 



THE CALCULATED RISK: AN ASPECT OF SCIENTIFIC PERFORMANCE 



David C. McClelland 
Wesleyan University 



S--Some of you may wonder how what I have to say is related to what Dr. Guilford 
has been talking to us about. I myself wondered for quite a while about it, and halfway 
through it occured to me that there was really a meaningful transition that could be made 
from his paper to mine. You could say by way of oversimplification that what he is in- 
terested in is what people can do, and what I'm interested in is what they do do. It 
doesn't follow that because a person can make very good systematic comparisons, for 
example, in one of Dr. Guilford's tests that he does this in real life very often. He may 
not be interested in this type of activity at all and this is where motivation comes in. 

A very crude analogy occurred to me which really is not very apt, but it's such an in- 
teresting analogy that I have to give it to you anyway. Suppose we were to measure the 
beer -drinking capacity of everybody in a group to try to use that as a predictor of al- 
coholism. In other words, would it be fair to predict from the fact that a person could 
drink a lot of beer that he would drink a lot of beer? Now, this is not a fair comparison 
because I believe that the capacity to do things is positively correlated with the tendency 
to do them in most cases . Whereas this may not be true for beer drinking, I believe it 
is true for many of the things that Guilford has been talking about. In other words, in 
all of Guilford's tests the subjects show what they can do when they are specifically asked 
to do a particular thing. This does not tell us directly whether they do these things or- 
dinarily. It shows that they can do them if they are asked. And this is one reason why 
I'm not terribly interested, by the way, in teacher judgments or grades even as an inter- 
mediate criterion, because all too often such criteria also merely show whether students 
can do certain things when they are specifically required to do them. Yet very often, es- 
pecially when we are dealing with high-level scientific creativity, the real criterion is 
whether or not people create spontaneously whether they are asked to or not. 

The hypothesis that I want to discuss is an example of the title of my paper: it in- 
volves a calculated risk. * That is, I should warn you in advance that it involves some 
pretty speculative elements which go well beyond the proven fact . Yet while I regard 
it as a long shot, I do not regard it as a complete shot in the dark. It involves a risk, 
but I hope it will turn out to be a well calculated risk. And as you will see, it is my 
contention that scientists must not only be willing to take certain risks in their research 
plans, but must even to some extent enjoy or get satisfaction out of the risk-taking enter- 
prise. 



What brought this matter to my attention originally was a discussion I once had 
with a graduate student who had read up on all the literature on a certain subject. He 
knew practically every research result that had ever been reported on a particular topic, 
and after he had finished summarizing in a seminar the present state of knowledge in 

*This paper is designed as an example of how an hypothesis is formed, not of how 
it is checked. A follow-up study is planned to check the hypothesis. 



this area, I somewhat unthinkingly asked him what he thought ought to be done next. It 
was obvious that the question took him by surprise, and he finally stated that he was in 
no position to judge since authorities had differed on what the crucial variables were 
and much of the evidence was conflicting. I pressed him further. I pointed out that he 
must by now know as much about this field as anyone else and that he ought to be willing 
to make a decision as to what the most promising line of inquiry was. He still showed 
some unwillingness to do this and ended up by suggesting that perhaps a massive research 
attack on all fronts at once would pay dividends. I pointed out that the research design 
he had in mind in which one entertained all hypotheses at once and varied all possible 
variables simultaneously was really pretty impracticable and that it would take several 
lifetimes to carry out. Surely a good scientist would be the one who could make the 
right decision as to what the most promising approach would be, however he might be 
able to arrive at such a decision. 

This discussion then reminded me of several scientists whose careers had turned 
out to be quite different from what one would have expected when they were in graduate 
school. For example, there was the brilliant academic student like the one in my semi- 
nar who seemed to understand practically everything that there was to understand about 
any given scientific problem, but whose research productivity since graduate school 
had not been conspicuously distinguished. Perhaps he was the kind of person who 
couldn’t make up his mind as to what was the most promising thing to do and there- 
fore, had dissipated his energies in a variety of enterprises, none of which he dared 
invest enough of himself in. On the other hand, bcould think of individuals who had 
not been particularly outstanding for brilliance or fluency of ideas in graduate school 
but whose subsequent research activities had exhibited what their friends were apt to 
call an amazing piece of luck. ” That is, they had hit on some promising line of re- 
search and managed to make a contribution to the field. In other words, they had in- 
vested wisely what time, energy, and resources they had in some line of endeavor 
which paid off. 



Thinking about such matters as investing one’s time and energy wisely and taking 
well calculated risks that paid off reminded me of another type of occupation in which 
these elements also play a large part - -namely, the occv /ation of the business entre- 
preneur.. Could it be that the scientist and the entrepreneur have something in common? 
For one thing, it is widely accepted, and I believe with a considerable basis in fact, 
that Protestant homes produced an unusually large number .of both scientists and suc- 
cessful business men. The argument and the supporting figures have been presented 
by Knapp andGoodrick, Parsons, Max Weber, Merton, and others. The two occupa- 
tional types would seem at least to have a common source --namely, the Protestant 

character structure, not in any strictly religious sense, but as it has been defined by 
sociologists. 

But at the surface level of occupational interests, these two types of occupations 
are almost diametrically opposed as Table 1 shows. In this Table I have simply taken 
the correlations of several occupational interest scales from the Strong Vocational In- 
terest Blank to show that sales occupations and scientific occupations are almost per- 
fectly negatively correlated. How, then, can they have anything in common? 
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Table 1 



Intercorrelations between representative scales 
and scientific interest scales from the 
Strong Vocational Interest Blank 
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At this point the cautious scientist (and perhaps the wise one!) would give up and 
say that obviously something is wrong with the whole analysis and that the reasoning to 
date is incorrect: Perhaps, for example, the association between Protestantism, science, 
and business enterprise is somehow accidental and in no way related to basic personality 
structure. Some of you may drop dff the train at this point, and I, for one, cannot blame 
you. I, myself, went further, I think, because I was too curious to stop. At any rate, a 
possible explanation for the lack of correlation between these two types of occupational 
interests might be that at some deeper or unconscious motivational level they are in fact, 
similar, although at the surface level of conscious interests they might be completely 
dissimilar because of some experience that one group had had which managed to deflect 
interests in the opposite direction from those of the other group. A particular possibil- 
ity occurred to me: Perhaps they both were high in what our research group has been 
studying for some eight or nine years now -- namely, the achievement motive (abbrevi- 
ated "n Achievement" for "Need for Achievement"). Now, as many of you know, a per- 
son is not aware of how much n Achievement he has. We measure it indirectly from 
his fantasy production and it has almost no correlation with his conscious desires for 
achievement (see The Achievement Motive , by McClelland et al) . Thus, it qualifies 
as a deeper -lying or unconscious motivational factor which the two groups might have 
in common without knowing it. 

But it qualifies for several other reasons. For example, we have by now collected 
a considerable amount of data showing that Protestant parents, particularly emphasize 
independence training for their sons (see McClelland, Rindlisbacher, and deCharms), 
that independence training produces higher n Achievement (see deCharms et al. ), and 
that higher n Achievement is associated with an interest in certain risk-taking entrepre 
neurial activities in the business world. For instance, we have just been able to show 
that subjects who are high in n Achievement - - whether they be male college freshmen 
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at Wesleyan University, high school students in Utica, New York, or in Mainz, Germany — 
indicate a greater liking for the entrepreneurial occupations of "stock broker" and "ad- 
vertiser. I should also emphasize that this preference for stockbroking exists despite 
the fact that, especially in our German sample and to a lesser extent in the American 
college sample, the occupation of stockbroking is not liked by the majority of students. 
In other words, this is not a case of people with high achievement motivation simply 
preferring somewhat more what their peers in general regard as a high-prestige oc- 
cupation. Also, the differential between those with high and low achievement motiva- 
tion does not exist for other high -prestige occupations like law, chemistry, and the 
like. 



Instead, 1 would like to argue that the reason why subjects with high n Achieve- 
ment are particularly interested in such occupations is because of their risk-taking 
character in which die success or the failure of the enterprise depends to some extent 
on the skill of the person who is taking the risk. Atkinson, at the University of Michigan 
has just reported some preliminary data showing that subjects with high n Achievement 
do in fact work harder under somewhat longer odds than do subjects with low n Achieve- 
ment. That is, the "highs" work hardest when the odds are one in three that they can 
win and work less hard when the odds are either longer (e.g., one in twenty) or shorter 
(e.g. , three out of four). The "lows" work harder at shorter odds where there is more 
certainty of winning. The explanation that he gives for this is either that the "highs" 
get a greater subjective feeling of success if they win when the odds are longer or that 
their confidence in their ability to succeed is such that their own subjective odds are 
actually shortened from, say, one in three to one in two. In either case the argument 
is the same so far as we are concerned: n Achievement attracts people to certain types 
of calculated risk-taking activities, in which the risks are neither too great (where suc- 
cess would appear to depend largely on luck) or too small (where anybody could succeed 
since it is a sure thing). So we have some highly tentative findings which do provide a 
little indirect support for our hypothesis that n Achievement may pre -dispose people 
to certain types of risk-taking which are involved in business activities and perhaps also 
in science. In short, the interest in taking well calculated risks to produce a feeling of 
personal achievement may be the factor which business men have in common with scien- 
tists . Unfortunately, at this moment we have no direct evidence on what would be the 
simplest test of all. We do not know whether successful entrepreneurs and outstanding 
scientists score higher on our test of achievement motivation than do men in general. 
Research is underway which will provide data for such a direct test of the hypothesis 
and, of course, it may very well reveal that my speculations to date are unfounded: 
this is what I mean by taking a risk. In this case I am even taking a risk in public. 

But I decided to hedge my bet as best I could in the time available with data al- 
ready at hand. It occurred co me that while scientists and sales occupations might be 
very highly negatively correlated on the Strong Vocational Interest Blank, there might, 
nevertheless, be a few items which would be scored positively for both types of occu- 
pations. To check this possibility I took the Life Insurance Salesmen and Real Estate 
Salesmen answer keys and compared them with the answer keys for mathematicians, 
chemists, and engineers. I noted down all the items out of a possible 400 which were 
scored plus 1 or more for each of the occupational scales, indicating that men in each 
of these occupations chose that answer more frequently (p<07) than did men in general. 
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There were not very many such items as you can imagine, since the total keys are so 
highly negatively correlated. The results showed that about all these two sets of occu- 
pations had in common was a dislike for routine jobs like bookkeeping and for certain 
artistic occupations and activities such as being a poet. In fact, things didn't really 
begin to get interesting until I also added the results of another study we had made in 
which the preferences of subjects with high n Achievement were compared with those 
of low n Achievement on every one of the 400 items in the Strong Vocational Interest 
test. When this analysis was coxnpleted - -that is, when the preferences of the two sales 
occupations, of the three scientific occupations, and of the subjects with high n Achieve- 
ment were compared -- only two or possibly three items out of 400 were answered con- 
sistently throughout all groups . In the first place, they nearly all disliked the occupation 
of being a poet. We need not make a great deal of this since it is not directly relevant 
to our argument, but it is perhaps worth noting in passing that there is definite support 
here for the notion that none of these people are introverts in the sense of being highly 
interested in their own internal thought processes. On the other hand, they have a strong 
outward orientation toward the world which involves things as far as the scientists are 
concerned and people as far as salesmen are concerned. The other item on which there 
was fairly complete agreement was Item 399 which gives a person an opportunity to say 
whether he frequently, occasionally, or never makes wagers. The scientists, the sales- 
men, and the subjects with high n Achievement, all (with the exception of the mathema- 
ticians) stated that they frequently or occasionally made wagers as opposed to never 
making them. Obviously, when this result popped out of the hopper I began to sit up 
and take notice. Here again was the indication of common interest in risk-taking situa- 
tions. Incidentally, the one other item on which all the groups (again with the exception 
of the mathematicians) showed some agreement was an interest (or not a positive dis- 
like) for playing bridge, a fact which lends support to the notion that what all these people 
have in common is a liking for situations which involve calculated risks. 

But before we go too far too fast, there would seem to be some flies in the oint- 
ment. There are other risk-taking enterprises mentioned in the Strong Vocational 
Interest Blank such as "auctions, " "bargaining. " "meeting new situations, " "fortune - 
telling, " which do not show the same results. How come? Two factors obviously need 
to be taken into account: in the first place, as far as the scientists are concerned, the 
risk-taking must not involve an interpersonal situation. Much /has been said at the level 
of popular folklore and also at the level of careful case studies by Anne Roe (Psychol. 
Monogr. , 1951) about the social inadequacies of scientists. They seem to have difficulty 
in interpersonal situations and to avoid them wherever possible . The evidence from the 
Strong Blank is very clearcut on this point. Of the 72 answers which are scored plus 1 
or more icr the three scales of mathematician, chemist, and engineer, at least 47 in- 
volve a clear dislike of interpersonal relationships or a clear preference for solitary 
activities. For example, scientists would much prefer beinga lighthouse tender to being 
a head waiter (Item 324). They like taking long walks and dislike bargaining and all sales 
or organizational activities. Thus, any risk-taking activities such as auctions, which in- 
volve interpersonal relations, will be knocked out in the scientist scales because of a 
powerful social inhibition. The risk-taking must involve nature rather than people. (See 
Table 2. ) 
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In the second place, the risk-taking must not involve simple luck, if our analysis 
above is correct that one of the reasons why subjects high in achievement like risk- 
taking is because it gives them a greater subjective sense o r success if they do win. 
Thus, it is not surprising if we do not get results for items like ’’fortune -tellers" or 
’’taking a chance vs. playing safe’’ (Item 334). In fact, the more I think about it, the 
more I realize that it was probably largely luck that there was agreement for the item 
about making wagers, because it is also quite evident that these same people have a 
strong prejudice against gambling. That is, we have argued above that scientists and 
business entrepreneurs come more often from a Protestant background, that they are 
interested in risk-taking enterprises, and yet at the same time it is well known that all 
Protestant churches have been strongly opposed to gambling. Right in my home state 
at the present time there is a political fight going on over legalizing gambling for church 
lotteries in which the opposition appears to be almost wholly Protestant whereas those 
favoring the bill are largely Catholic. How come? The point seems to be that it is pre- 
cisely because the Protestant is especially drawn to "proper" risk-taking that he must 
enforce a prohibition against "improper" risk-taking (see Devereux). 

The difference in some instances is obviously one of degree, but the crucial fac- 
tors are (a) how long the odds are, and (b) how much depends on the person's own know- 
ledge and skills and how much on luck or forces outside his control. So far as the 
Protestant is concerned (or the scientist or the business entrepreneur) he is caught in 
the position of wanting some risk to get some feeling of personal satisfaction for suc- 
ceeding, but a risk which is not too great and not too dependent on forces outside his 
personal control. Thus he disapproves or outlaws gambling to make certain that people 
are forced to make a discrimination between two types of activities which are not always 
easy to distinguish. Thus, if the Strong Interest item had been in terms of frequently 
or occasionally or never placed bets , I am sure the results would have been quite dif- 
ferent. The fact that the term "wagers" was used left open the possibility of the re- 
spondent’s thinking in terms of relatively harmless wagers about how things are going 
to turn out which don’t fall into the clearly disapproved category of gambling. 

To summarize, I have tried to make a case on the basis of some suggestive but 
hardly conclusive evidence for the hypothesis that the successful scientist, like the suc- 
cessful business entrepreneur, is someone who will have a higher n Achievement, in 
part because high achievement motivation will predispose him to take moderate calcu- 
lated risks in which the success or failure of the enterprise will depend on his own 
efforts . If the hypothesis is to be taken seriously — if it is a sufficiently good risk to 
be worth the investment of anybody's time, energy, or other resources — then there 
are some obvious next steps which should be taken: 

(1) A direct test of the hypothesis should be made to see if highly successful 
scientists and business entrepreneurs do in fact have higher n Achievement. 

It is probable that it is only a small minority of really creative scientists 
who have high n Achievement, because it appears that science is not an oc- 
cupation which is particularly attractive to those with high n Achievement, 
possibly because many scientific functions can be performed with no risk 
and no chance for creativity whatsoever. 




101 



(2) Whether or not (1) turns out to be correct, it would seem to be worthwhile to 
study the way scientists make decisions as to what it is worthwhile to investi- 
gate and the subjective feeling of risk involved when they make such decisions. 

(3) Further research on the characterological difference between scientists and 
business entrepreneurs is clearly necessary. If it is true that they both have 
had a background of independence training and high n Achievement, it is equal- 
ly certain that something has happened to the scientists which has made them 
withdraw from interpersonal contact and work out their energies at the level 
of material, inanimate objects, (see Table 2) whereas the business entrepre- 
neurs are turned in the opposite direction toward an interest in contact with 
people. Incidentally, the salesman's interest in people does not go so far as 
wanting to nurture and help them. Neither he nor the scientist, for example, 
liked the occupation: of Y. M. C. A. worker. There is support here for the no- 
tion that the scientist is interested in manipulating things and the entrepreneur 
in manipulating people as things, but the factor or factors which are respon-. 
sible for this "manipulatory attitude" are, so far as I am aware, unknown and 
should be investigated. 

C — May I offer one correction — the correction to the effect that I’m not interested 
in what people do or what creative scientists do. I am interested in what creative scien- 
tists do. And I recognize that if you hold aptitudes constant, surely motivation will have 
probably the bulk of the determination of the output. I hope too that McClelland will also 
recognize that if you hold motivation constant, aptitude will have something to do with 
performance. 

S--Sure. I was carried away by the moment and I entirely agree with you, Dr. 
Guilford. 

C— I thought that you were brushing away aptitudes entirely. 

S— No, I didn’t mean to, I’m sorry. The example of alcoholism was not really 
any good but 

C--I accept that! (laughter) 

Q — Would you say that the dislike that the man interested in calculated risks had 
for dealing with another individual when he takes his risk would preclude taking high n 
Achievement people and putting them into experimental situations where they literally 
wager for us in partial reinforcement studies, aspiration studies, or what have you? 

S--Well, we do know this. People high in n Achievement don’t like to place bets 
on the~chance roll of a die. They don’t want to take a risk here because they don’t have 
any control over the situation. 

Q--Yes, they want a problem where they have some semblance of control. You 
could structure it that way you know, where they have such possibilities. Would the 
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presence of the experimenter in such a case preclude their getting involved? 

S — I don't know. We’ve never tried it. 

C— It seems to me that experimentally some work could be done with it so as 
to avoid the whole problem of cutting into business and scientific life. Coming to 
grips with wagering in real life is a slippery variable to get hold of. 

Q~I heard a radio report last week that reminds me of your talk in which a 
person of a foundation was asked the question, "Why didn’t his foundation invest more 
money in things that were more a hundred to one shot, ” and his defense partly was: 
’’They thought they did. " They gathered groups of scientists around who advised them, 
etc. , and they thought they were doing this. Are there people who take these twenty 
to one shots? Is a scientist this kind of person? Is this one of our problems? Are 
there enough people who are willing to bet on such long shots? 

S— It’s the one in three shots that probably pay off --not the one ia twenty. 

<C Well, one in twenty or one in a hundred is what we ideally might want. 

JSh- Foundations do try to back long shots often. 

"Yes, but are there such people around in science? Do we have individual 
differences so there really are such people? 

S--You mean people who want to take the one in twenty shot? Oh, I think so, 
but I think by definition they’re not apt to be the successful ones. 

Q — Are they the ones with such high self confidence that the odds are subjective- 
ly reduced to less than twenty to one? Couldn't the ones who have performed the ex- 
periment in their heads before have become convinced that what is generally perceived 
as a one in twenty shot, may be a fifty-fifty shot? 

S--Yes, that may be it. 

C--I can think of a particular scientist I know who likes to take long shots on the 
races, but he has gone to a great deal of work to decide that he is getting a fifty-fifty 
break or something approaching that. It's the cutting down the ostensible odds to what 
he thinks are the real odds and then putting his bets there that he enjoys . 

C--I know one who does that, and he does beat the odds. 

C- -Maybe you and I know the same person! 

C- -But the whole problem here is that they are not blind odds, but odds in which 
rational control or calculation enter in as a factor in your favor . 

“Well, how about the Strong? Are there other items in the Strong that reflect 



this "pure luck” sort of thing? 



S--I've already mentioned a couple of such items. For example, preference for 
taking risks vs. playing it safe doesn't show a difference between those with high and 
low achievement motivation. 

Q- -Have you carefully scrutinized the Strong to see if there are, other items dealing 
with pure chance? 

S--I think that I have looked through everything. I hope that you realize that I really 
don't know whether to take this hypothesis seriously or not. I don't know what the odds 
are. It is the kind of hypothesis I like, that's obvious. I can see why other people wouldn't 
like it because it involves a whole lot of fairly subtle analyses of different things . 

Q — Actually, are we speaking here about the calculated risks for scientists and whe- 
ther they follow this particular line or that one . How often can you actually calculate the 
risk? Decisions have to be made blindly. 

* 

S- -That’s right and that's what I meant when I said we ought to do a little studying 
about what goes on in a scientist's head when he sees a range of possibilities and says, 

"I feel that this is the thing to do. " Suppose someone says that that has been tried. What 
then? Does he go ahead anyway? What happens in a person's mind, for example, when 
the result doesn't come out as he expected it to come out? One man might say, "Well, I 
guess I was just wrong, I don't know what was the matter. " Another might refuse to give 
up and say that the data were wrong. Now, the point that I'm getting at is that somehow — 
and I don't quite know how to do this - -we have to distinguish between those who say the 
data are wrong when the data are right, and those who say the data are wrong when they 'ire 
right. This is the essence of scientific performance - -that is, successful performance -- 
and it is exactly similar, as far as I can see, to what the business entrepreneur has to 
face when he says, "Well, will I build an addition to the Alpine Rose Lodge, or won't I? 

It'll cost me so much money, and is it a good investment dr isn't it a good investment?" 

If he makes the wrong decision, then he loses money,, and he may have to go out of busi- 
ness or something. The same happens to the scientist, although some of them already 
have tenure! 

C--Well, there are three variables ordinarily in a gambling situation. One is the 
expectation of gain which would be like your achievement motive . Another is the risk of 
ruin and the difficulty here is defining psychologically what ruin is when money is not in- 
volved. When money is involved, it is simply the ratio between what you stake on each 
individual event and the total capital you have. And the other variable, of course, is the 
probability that the event will occur. So you can't merely deal with what the odds arethat 
the discovery will be made or that the hypothesis will turn out to be correct. But you also 
have to consider how much of the person is being invested in this gambling action and what 
would ruin consist of. And with those three, then you can conceptualize the problem ex- 
perimentally. For example, there is no reason why you can't, if you want to do this, 
equate n Achievement to "expectation of gain" and vary the other two things experimen- 
tally. 
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S ' “Well, as a matter of fact, we have independent ways of measuring fear of fail- 
ure and hope of success, and it is my guess that the person who does the safe thing -- 
who let's say, just adds a footnote to the great master's design under whom he studied 
in graduate school --is the high fear -of- failure person. Perhaps he doesn't risk much, 
and there is a very good chance that something will come out no matter what he does. 
Then, at the other extreme, you have the wild -eyed people, the ones who like penny 
uranium stock - -there are scientists like that, too, you know - -full of wild ideas. I 
don't think that they get anywhere either, by and large. It is the people who fall some- 
where between these who, I think, make the biggest contribution creatively to science. 

C_- “Well, it's a minimax problem, and presumably the person who takes calculated 
risks well is he who has solved the minimax problem through these three variables. 

S- -Knapp and I have talked about - -it was Knapp's idea originally - -the possibility 
of checking this ability through some kind of test which asked persons to make estimates 
about problem situations which they would not ordinarily have specific information on 
such as the distance from here to Kamchatka. Maybe this is the kind of guy who could 
size up situations like this and come closer to the truth with a good estimate. 

C--Thurstone has tried some tests like this - -guessing or inspiration tests. 

C--What you are talking about involves a choice point where a decision has to be 
made either by intuition or by strong rational control. 

S--Well, it isn't really a choice between intuition and reason but a successful com- 
bination of both - -a combination that possibly these people with high achievement motiv- 
ation may have developed highly in all spheres of life from early childhood. The hope 
is that you could tap it in some way by a test. 

2" "You mean decision by intuition? 

§ Correct decision by intuition - - intuition meaning the unconscious weighing of 
the information and coming out with a good synthesis somehow. But again, even if you 
could measure it with a test, I'm not sure that the people who scored high on the test 
would necessarily be the ones who would do it spontaneously. 

C--We have done some work with Thurstone's hypothesis which I will report on 
and I can tell you that they can do it, that you get considerable individual differences 
and that some of them stumble along and never know whether they are right or wrong 
and yet give you responses beyond chance . 

(J--What do you have in terms of a conceptual scheme? What do you have in mind 
for n Achievement? In the midst of your talk I suddenly had the feeling that it was be- 
coming a simple and sovereign solution to a whole variety of problems, and I saw that 
you weren't thinking that way. But conceptually speaking, how do you conceive of it? 

S--Well, at the present time I mostly conceive of it operationally, that is, by the 
way we measure it and by its correlates, and by what produces it. It is not the same 



thing, for example, as what Murray means by n Achievement. 



Q-- Could I ask a question about the sociological concept of Protestantism? That 
one worries me a little. In the first place, I think Max Webejr's thesis has been kicked 
around a bit. Some people think that people who are entrepreneurs tend to become 
Protestants and others think that people who are Protestants tend to become entrepre- 
neu. 3. I wonder if you look at this thing in terms of father's occupation, in terms of 
the . ccupational orientation of the family, whether Protestantism might make less dif- 
ference. Certainly, if you discriminate between a Protestant group and a Catholic 
group' you probably will find differences in occupational orientation. But there is still 
the question, "How Protestant are these people? How much does Protestantism have to 
do with the way they see things?" 



Q- -Aren't you using Protestantism as defining a general cultural system? 

S--A value system, yes. T don’t refer to whether they go to church or not. They 
may be quite irreligious, althougn they may accept the ethic of Protestantism, including 
such variables as postponement of gratification, or willingness to renounce immediate 
impulses for long-range gain. In other words I am using the term Protestantism not in 
a theological or sectarian sense, but in a sociological one as referring to a group of 
people brought up culturally as Protestants who may or may not be formally religious. 
Well, I don't know the answer to your question in so many words. It's a kind of chicken - 
and-egg question, a little like the heredity -environment controversy. We hope to get 
some data on it by time series. We're doing some time series now to see whether the 
ideology changed first, or the economic activity changed first. That is, do certain 
values and motives lead to certain kinds of economic activity or occupational orientation 
or is it the other way around? All we can say now is that they go together, but we don’t 
know which determined the other. 

C£--Why do you select the Protestant ethic rather than social class as your major 
explanatory variable? 



C--I would like to add the comment to that question that I've heard people who 
are experts in comparative religion maintain that the basic ethics of all great religions 
are almost identical. 



S--But they aren't. 

C--That's what you say, but as compared with theology, they are very similar. 

S--Well, it just isn't so at the folk level. At the level of high religion it is prob- 
ably true, but at the level of folk religion, it is not true. There is great dissimilarity 
in the .beliefs of the common people among different religions as our questionnaire 
studies have shown. A number of Irish mothers think, for example, that it is effica- 
cious to pray for your child. Most other people just don't think so — even Italian mothers 
who are Catholics, too. In this caste, there is some ethnic factor that comes in. But 
at the level of folk religion, that is what people really believe, there are major differ- 
ences in values shared and ethnical beliefs . But by the time it gets to the philosophers 
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and mystics, I agree that they make it all sound the same. 

Q- -Aren’t there other cultural matters than religion which are better focused 
perhaps? 

SI- -Well, perhaps. I don’t really know. I might say at this point that a lot of my 
thinking on these value questions was stimulated by the work of one of the men on the 
SSRC committee on identification of talent, Dr. Fred Strodtbeck, who did a major re- 
search project which will be reported in our monograph on value differences promoting 
achievement. He contrasted Italian and Jewish culture in New Haven in value terms and 
found that there were really major differences in the orientations of these two groups of 
people. 

-Were these equated for social class? 

S--Yes, they were matched for social class. He had a factorial design in which 
there were under-and -over -achievers, and high, middle, and low socio-economic status, 
so that you could pull all of these things out separately. 

<C--There are some quite complex systems for analyzing value systems, like Flor- 
ence Kluckhohn’s and Talcott Parsons’ which might be more discriminating than just this 
idea of Protestantism . 

✓ 

S--Oh, we use their value schemes . In other words, I’m just using Protestantism 
as a generic term to describe a value system. In fact, Wilson is using in Portland some 
of the values scales we have developed. But as a matter of fact, one of the very striking 
things about such data, if you look into them, is that there are large differences in occu- 
pational achievement of, say, Italians and Jews in this country. It’s been common to ex- 
plain these away as being due to accidents, or perhaps to race prejudice. The argument 
may run that the Jews are discriminated against, they counterstrive harder, and, there- 
fore, they succeed more . But if you use that argument consistently, you would have to 
argue that whenever Catholics are discriminated against, they should work harder too, 
but they don’t. On the other hand, if you look into the value orientations of Jewish culture 
and discover the stress their religion places on knowledge, and the importance of know - 
ledge, the whole thing becomes quite clear. There is a curious unwillingness among so- 
cial scientists, from my point of view, to believe that what people think is important -- 
that is, basically, religiously important - -is going to influence what they do. Suppose 
a family believes that learning is really one of the highest goods - - that to have a scholar 
in the family is to reflect credit on everyone in it. Is it so surprising that they are go- 
ing to make all kinds of sacrifices to see that their children go to school, and help them 
out in any way they can? Yet this is often what Strodtbeck found in his Jewish families 
and not in his Italian families. Such a difference should make a tremendous difference 
in their future careers, in their occupational advancement. Well, that’s one of the 
things that came out of our talent committee work — the importance of cultural values 
in talent development. Since you have been interested in it, I might mention the two 
other things that our committee did work on just in case anyone wants to ask me about 
them afterwards. Most generally we were interested in non -intellectual determinants 
of achievement (e.g. , values) and in non-academic criteria of achievement. The two 
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non -academic types of achievement we sponsored research projects on were 1) social 
sensitivity, which turned out largely to be a study of methodological errors which in- 
validate much of the work in this area; and 2) a study of community service activity 
as a criterion of success in small towns. We also sponsored a theoretical paper by 
Baldwin in which he tried to integrate test -taking behavior into the general theory of 
behavior. It is his feeling that test -taking behavior has not really been integrated into 
the general theory of behavior. I won't attempt to characterize his article except to 
say that I think that it is extremely important and that some of the things I said very 
hastily this afternoon about the difference between what a person can do and what he 
does do were drawn in a very crude way from his paper. 



Table 2 



Items scored +1 or more on the Strong Vocational Interest Blank for the 
Mathematician, Engineer, and Chemist scales indicating men in these 
occupations chose these answers more often (pc^.07) than men in general. 

A) Avoidance of interpersonal contact (N = 47) 





Like 


Dislike 






142 


Taking long walks 


2 Advertiser 


197 


Intel . iewing pros- 


314 


Member of a society or 


8 Auctioneer 




pects in selling 




club (forced preference 


11 Auto Salesman 


200 


Organizing a play 




if they do not wish any 


18 Buyer of mdse. 


217 


Bargaining (’’swap- 




leadership positions in 


38 Floorwalker 




ping”) 




the club --see 317, 320) 


42 Hotel Keeper or manager 


219 


Buying mdse, for a 




Prefer 


47 Judge 




store 






51 Lawyei, Criminal 


220 


Displaying mdse, in 


324 


Lighthouse tender to 


52 Lawyer, Corporation 


229 


a store 




headwaiter 


54 Live Insurance Salesman 


Raising money for a 


326 


Gardening to house to 


70 Playground Director 




charity 




house canvassing 


72 Politician 


287 


Selling a machine 


328 


Develop plans to execute 


77 Real Estate Salesman 


288 


Preparing advertis- 




plans 


78 Reporter, general 




ing on a machine 


331 


Deal with things to deal 


80 Retailer 


290 


Interesting the public 




with people 


81 Sales Manager 




thru speeches 


347 


Technical responsibility 


85 Secy. Chamber of Com- 


310 


Being John Wanamaker, 




to supervisory responsi- 


merce 




merchant 




bility 


89 Social Worker 


317 


Being Chairman, En- 


355 


Few intimate friends to 


90 Specialty Salesman 




tertainment Comm. 




many acquaintances 


92 Stock Broker 


320 


Being Chairman, Pub- 






95 Traveling Salesman 




licity Committee 




No 


99 Wholesaler 






365 


Usually liven up group 


100 A worker in YMCA or 








on a dull day 


K. of C. 






37* 


Remember faces, names 


110 Dramatics 








and incidents better than 


131 Public Speaking 







the average person 



B) Manipulation of things (N = 24) 

Like 



6 


Astronomer 


101 


Algebra 


148 


Solving mechanical puzzles 


10 


Author of Technical 


107 


Calculus 


186 


Repairing a clock 




book 


108 


Chemistry 


212 


Doing research work 


24 


Civil Engineer 


114 


Geology 


281 


To develop theory of operation of a 


32 


Electrical Engineer 


115 


Geometry 




new machine, e.g., auto 


45 


Inventor 


120 


Mathematics 


283 


To discover an improvement in the 


83 


Scientific Research 


128 


Physics 




design of a machine 




worker 


144 


Chess 


303 


Being Thomas A. Edison, inventor 



a Except Mathematicians (items of special interest discussed in the text) 



Table 2 (cont'd) 



Prefer Yes 

328 Developing plans to executing 368 Have mechanical ingenuity 

plans (inventiveness) 

331 Dealing with things to dealing 
with people 

347 Technical responsibility to 
supervisory responsibility 



C) Order and security (N = 5) 
Like 

223 Methodical work 
Yes 

378 Can write a concise, 
well -organized report 



Prefer Dislike 

335 Definite salary to 92 Stock Broker 
commission on work 
done 

338 Working in large or- 
ganization to working 
for self 



D) Independence (N = 5) 



Dislike 

64 Office Clerk 

65 Office Manager 
74 Printer 

243 People who assume leadership 
296 Opportunity to ask questions and 
consult about difficulties 

E) Aesthetic and recreational interests (N-7) 



Dislike 



Like 



71* Poet 

153 Amusement Parks 
194 Decorating a room with 
flowers 



146* Bridge 
164 Museums 
167 Symphony Concerts 
181 "Atlantic Monthly" 



F) Risk-taking 



Dislike 



399 Frequently or occasionally 92 Stock Broker 
make wagers a 



a Except Mathematicians (items of special interest discussed in the text) 
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SOME PROBLEMS IN IDENTIFYING CREATIVE 
SCIENTIFIC TALENT AT VARIOUS ACADEMIC LEVELS 

Henry S. Dyer 
Educational Testing Service 



S--When we talk of identifying creative scientific talent in anyone who has not yet 
completed his formal schooling, what we mean is that we wish to siDg’e out those indi- 
viduals who, with proper training and guidance, will have a good chc.nc.3 of becoming 
creative scientists . The main reason, I suppose, why we want to single out such indi- 
viduals at any academic level - -and the earlier the better -- is to see to it that they are 
encouraged and helped to get the kind of schooling that will make them what we think 
they are capable of becoming. 

As things now stand, there are three types of problems that prevent us from car- 
rying out this task of identification as well as we should like. The first type of problem 
is an intensely practical one: we have no handy system for reaching all the youngsters 
in all the schocls to see which ones might be showing the future promise we are looking 
for. There is no truly national testing program, for example, of even the crudest and 
simplest sort which could conceivably provide a reasonably complete census of the un- 
usually bright. I don't believe it is generally recognized that in spite of all the testing 
programs combined, whether sponsored by states, counties, municipalities, individual 
schools, or private organizations, only a fraction of the total school population is 
reached. The more remote schools that contain a large minority, possibly a majority, 
of the student population either cannot afford or have no interest in even a minimal pro- 
gram whereby it might be possible to separate out the abler pupils who would consti- 
tute a general pool from which potential creative scientists might be drawn. Even if 
one were to write off this large unreachable group, one still has to face the fact that 
the tests which do reach pupils are sponsored by so many different and uncoordinated 
bodies, official and otherwise, that there is no way now in sight of making sure that 
the results can or will be used to sift out those pupils who should receive special at- 
tention. 

The second problem has to do with the kinds of tests and interpretive data avail- 
able to those school authorities who might be interested in making a serious effort to 
locate pupils who have the best chance of becoming creative scientists. I think it is 
safe to say that at the present time there are no tests backed up by sufficient data to 
enable a guidance counselor to say with any real assurance of either a high school fresh- 
man, or for that matter, a college freshman, that he is more likely to succeed as a 
scientist than as something else. The reason for this is that the necessary research 
to secure the data has not yet been donr . 

We know in a' rough way from Ter man's studies, of course, that a youngster with 
a high IQ has a better chance of becoming a productive intellectual worker than has the 
youngster with a relatively low IQ. But I am aware of no long-term developmental 
studies of any consequence that have established the various factors which enter into 
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different types of professional success -- that is, studies that begin with the child in the 
elementary school and follow him up until he is established in a career. I suppose this 
is because the long-term studies of mental development were begun at a time when the 
psychologists who did them were interested primarily in g. 

The thousands of prediction studies that have been made over the past thirty years 
are not very helpful either. Most of them have been concentrated at the college level, 
have been short range, and have taken such a shotgun approach that the results are 
scarcely generaiizabie. Brandwein’s study is the nearest approach i know of to a use- 
ful study aimed particularly at the prediction of scientific talent. But it is a very small 
study . It purports to predict college achievement in science from data gathered on a 
selected sample of tenth graders in a couple of New York City high schools . Brandwein 
claims that he can now predict pretty accurately who will make a creative scientist and 
who will not. Among the instruments he used, the one that seemed most effective for 
the specific purpose was a sort of behavior check list that served as the basis for rat- 
ings. I doubt whether the study in the form it took could be cross -validated by someone 
other than Paul Brandwein. 

From the standpoint of a long range prediction of special abilities, the factor an- 
alysis studies to date are of only indirect help. They describe the abilities that may 
underlie various kinds of achievement, but they have paid little attention to actual criteria 
of performance and practically no attention to how the abilities that are isolated may de- 
velop and change over a period of time, nor have they generally taken into account the ^ 
differential effects of training on the development of these abilities. 

I know of three studies, none of which is as yet finished, which may eventually 
make a small though probably inadequate dent in this problem of getting data for the pre- 
diction of scientific talent. The oldest is one that I began in 1946 at Harvard. I sur- 
veyed all the tests of special ability then current and settled on twenty-one measures 
that, in my judgment, seemed to cover all the important abilities necessary for success 
in each of the major fields. To this battery I added the Kuder Preference Record, and 
I administered the whole concoction to as many freshmen as would take it. As a size- 
able portion of every freshman class since then has been swallowing the same dose, a 
rather large amount of data has accumulated, and a study of the three earliest groups 
is now nearly completed. The results should tell something, but not a great deal, about 
who is likely to succeed as a science major. The limitation in this first attack on the 
data is the fact that the study is being conducted by a doctoral candidate who is primarily 
concerned with making sense out of the multiple discriminant function. But the data are 
there and still accumulating for additional studies if anyone can find the time and money 
to make them. * Iy original hope was to use the material as a basis for determining rel- 
ative probabilities of success in each field, the principal criterion being honors taken at 
graduation. It is admittedly not a very sharp criterion for assessing creative talent in 
science or any other field, but it is the best we are likely to get unless somebody can 
get hold of the cases after they have entered the professions and find out what they are 
doing and how well they are doing it. 

ETS got a study under way three years ago which is similar to the one I have just 
described. It differs from the Harvard study in two respects: first, the tes*s used, 
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except for some specially constructed interest measures, are all purportedly pure fac- 
tor tests; and second, the guinea pigs are in a variety of colleges and universities -- 
about nine, I think. In this study we expect to use multiple regression, rather than mul- 
tiple discriminant analysis, and to get relative probabilities of success in each field. 

ETS is also just now initiating the same kind of study with high school freshmen. 



None of these three studies, of course, is aimed solely at the prediction of crea- 
tive scientific talent, but forecasting such talent is one of the objectives . Some of the 
tests are, we hope, measuring factors related directly to the creativ. process. It could 
be argued, I suppose, that studies of this general design, which seek to provide data for 
determining relative probabilities of success in a large number of fields, do not consti- 
tute the most efficient approach to the particular problem of predicting creative talent 
in science. My own feeling is, however, that a narrower design — one that concentrated 
solely on scientists - -would in fact be less efficient, since it would probably not tell us 
what qualities are peculiar to the youngster most likely to become a creative scientist, 
and it is precisely that information that we must eyentually have if we are to separate 
out such people from the general run of the gifted. 



The third problem having to do with the identification of scientific talent in the 
young is that of discovering what kinds of training are most likely to convert an ordinary 
mortal into a potential scientist. This is to say that the training problem is an integral 
part of the problem of identification. Let me illustrate what I mean by a hypothetical 
example. Two ninth -grade boys, A and B, take a course in general science. They are 
tested at the beginning of the course and again at the end. Both have the same initial 
scores but A shows a larger gain than B on the final test. Does the difference in gain 
mean that A has a better chance of becoming a scientist than B? Actually, we don’t know. 
The chances are that for some course with a specifiable quality of instruction, the dif- 
ference in A's and B's performance would be meaningful, while in other courses the dif- 
ference would not be meaningful . I know of no research whatever that would throw any 
light on this question. And this is a very real question, because many guidance counsel- 
ors tend to rely heavily on differential performance in end -of -course tests as predictive 
of future success without ever examining the assumption that performance in the course 
is related to performance in later life, and without ever bothering to determine whether 
it is the course or something else that is operative in producing the final scores. 

I imagine that nobody who has given die matter any thought thinks any more that 
creative talent is simply inherent within the individual. It is the outcome of certain 
kinds of forces that play upon certain kinds of individuals as they grow up. It is acquired 
from experience. The trouble is that we actually know very little about the character- 
istics of those youngsters who are likely to acquire it or about the educational or other 
experiences that tend to produce those characteristics. To me, one of the most disturb- 
ing things to come out of the Terman studies is the implication that those of his gifted 
students who became productive thinkers seem to have got there in spite of their school- 
ing rather than because of it. This scarcely proves that intellectual development is a 
matter solely of inherent ability and temperament, for the subjects that may have blos- 
somed because of appropriate instruction were either left out of the study, or, if they 
were in the study, were left unidentified. 



Of the three problems I have discussed, the first - -that of reaching the untested 
and of making some sense out of the scores of the large numbers vho are tested - -is 
not so much a matter of research as of formulating some sort of rational action pro- 
gram, using the instruments we now have, to try to identify the complete pool of the 
unusually bright from which potential scientists might be drawn. If we could find the 
means, I think even a crude attack on this problem would have an enormous impact. 
The other two problems - -those that have to do with pinning down for successive age 
groups variables of individual behavior and of training that are significant for predict- 
ing scientific achievement -- are matters for long range and comprehensive research 
of the longitudinal variety. 

To bring the problem into focus I should like to outline a type of study which I 
have not yet fully thought through. It is probably unrealistic and premature, but I 
think it represents the sort of thing that will have to be done eventually if we are ever 
to get a firm grip on techniques for identifying the youngsters who are most likely to 
develop genuine creative power in science. There are three principal questions that 
would have to be answered in deciding whether the time was ripe for the kind of study 
I have in mind: 

1. Do we have a sufficiently clear idea of what we mean by creative behavior to 
recognize it when we see it, not only in mature scientists but in school and college 
students as well? 



2. Assuming that the answer to question 1 is "Yes, " do we have enough reason- 
ably firm and testable hypotheses concerning the factors of ability and temperament 
that account for creative performance? 

3. Assuming that the answers to both of these questions are in the affirmative, 
do we have enough good ideas about how these factors might be measured at successive 
levels from childhood to adulthood? 

The proposed study is what might be called, in view of its general design, an 
overlapping prediction study. It would involve at the outset upwards of 20, 000 students 
at several grade levels, and it would observe a large number of aspects of their men- 
t** 1 growth, and the environmental factors influencing it, over a four -year period. The 
v ign is shown in Figure 1 . Measures would be taken on each group twice a year at 
the spots marked "x. ” These measures would consist of three types: (1) criterion 
measures of creative performance, (2) measures of ability and temperament designed 
to predict creative performance, and (3) measures of such influencing conditions as 
family background, type of instruction, quality of instruction, extracurricular activity, 
and the like. At the points of overlap, the measures would be identical for the adjacent 
groups. With the increasing age of each group, however, the superficial character of 
both the criterion and the predictor measures would probably have to change by degrees 
to keep them appropriate to the students’ level of development, but it would be necessary 
to ensure, nevertheless, that these measures were tapping the same fundamental factors 
all along the line. The analysis would begin with a series of correlations relating all the 
measures taken at a particular point to all the measures taken at subsequent points . By 
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combining the data on adjacent groups at the points of overlap, it should be possible to 
work out reasonabJy secure connections between the criteria at Grade 3G and beyond 
and all the other measures taken at any point below, all the way down to Grade 3. 

The numbers of cases for each group are sheer guesses as to what might be 
needed to ensure usable results . Possibly the early groups should be considerably 
larger than I have indicated in order to guarantee that there would be enough potenti- 
ally outstanding scientists in each to provide adequate data at the close of the study. 

The big bulge in Group 5 is there to provide a sufficient continuing sample in college. 

The number in Group 8 is not specified because I have no idea how many suitable grad- 
uate schools one might be able to dig up for participation in such a study as this. 

One feature of the study not indicated in the chart is the strategy that would be 
followed to get an estimate of the effect of different kinds of training on the criterion 
and predictor variables at each stage. Groups 1 to 5 would be spread over some forty 
to fifty school systems so selected that there would be as wide a range as possible in 
the kind and quality of instruction given in science and mathematics from Grade 3 to 
Grade 12 . 

Groups 6 and 7 would be spread over about twenty colleges selected according to 
the same principle. Observation teams would be sent out each year to assess the in- 
struction the students were getting - -not simply by scrutinizing course syllabi, but by 
visiting classrooms and sizing up the actual performance of teachers . 

In order to minimize the difficulty of keeping track of the subjects, especially in 
the transition from high school to college, the schools containing Groups 1 to 5 would, 
as far as possible, be selected on the basis that they serve significantly as feeder 
schools to the twenty colleges in the experiment . 

I would hope that if such a study were to be carried out, the kinds of observations 
to be made at each level would not be determined on the basis of any iron-clad assump- 
tion about the general pattern of mental development. One such assumption, which I 
believe has considerable currency and of which I am particularly afraid, conceives of 
individual differences among younger children as being accounted for entirely by an un- 
differentiated general factor (g) and that various group factors, such as verbal, spatial, 
numerical, etc. , begin to split oft from g at later ages. Such an assumption might be 
so restrictive as to prevent us from turning up important information about the creative 
process, especially in younger children. It should be treated as a hypothesis to be 
tested rather than as an assumption to be taken for granted. For instance, it is possible 
that if we were to introduce radically different types of training in the early grades, the 
variance due to g would be immediately and drastically reduced and the variance due to 
a number of other factors might be greatly increased. Or perhaps the reason that we 
have been finding nothing but g in young children is simply because we have not invented - - 
or at least have not used - -tests that measure anything else at this level. It seems to me 
entirely possible, even probable, that various factors associated with creativity are just 
as sharply definable in young children as they are in adults -- perhaps more so. There is 
also a good possibility that native creativity in the young is frequently extinguished by the 
demands for intellectual conformity that schooling and the culture impose. If this is so, 



then the design of the study should permit us to find it out and get some notion of what 
the remedy might be . 

The study I have described is an attempt to compress into four years a longitu- 
dinal study that would otherwise take sixteen. Provided certain problems can be 
solved, the advantages of this telescoped design over a straight-line design are fairly 
obvious. The practical possibilities of making a really extensive study of mental de- 
velopment over a four -year period would seem to be much greater than if the study 
were to drag on for sixteen years, since the chances of holding the interest of the 
authorities in the participating schools and colleges and of keeping together the origi- 
nal team of investigators would be higher. Furthermore, the results would be avail- 
able sooner, so that the hypotheses with which the study began would stand less risk 
of being outmoded by the time it was finished. Also the probable loss of cases for all 
reasons would be minimized and what loss there was would not be cumulative, since 
the losses sustained in any one group, except the oldest, would, in a sense, be made 
up by the introduction of a full complement of cases in the next higher group. At the 
close of the study, therefore, one should be pretty well protected against having an 
insufficient number of cases in the cells of primary importance. 

This, of course, is a mammoth study. It would probably cost two million or 
more to complete. It probably will never be done. The advantages of considering it 
in some detail, however, lie in the fact that it points up most of the problems that 
have to be solved before we can get from where we are now to the place where we can 
actually spot the students in school and college who ought to be scientifically trained. 

One such problem that is interesting and particularly important has to do with 
the question whether by the use of overlapping groups in the manner I have indicated, 
you can actually observe the characteristics of growth across the areas of overlap. 

I had assumed in my simple-minded way that the use of ordinary multiple regression 
all up the line and the pooling of data in the overlapping areas would serve the purpose. 
But I have been told that this won't work and that a much more sophisticated mathemati- 
cal approach to the situation is required. Dr. Saunders claims to have solved the prob- 
lem theoretically, and the design of the experiment is such, I think, that the assump- 
tions on which the solution is based would be met. 

Another important series, of problems has to do with the factorial composition 
of the measures used for the successive groups. I have indicated that although these 
measures will necessarily have to differ as one moves up the scale, we should never- 
theless be as certain as we can be that they are measuring the same fundamental fac- 
tors. The probability is that the surface characteristics of many of the measures will 
probably have to differ if they are to measure the same factors in the older and younger 
subjects. 

Let me give two examples of the kind of problem I have in mind. Keats, one of 
our psychometric fellows, has just completed a study based on Piaget’s hypothesis 
that as children grow older they tend to be better able to reason in formal as opposed 
to concrete terms . The hypothesis can be expressed in a fourfold table like this: 
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F- 



F+ 



C+ 



C- 



where C stands for a concrete item, F for a formal item, + for an item passed, and 
- for an item failed. The hypothesis is that when both items are presented to a group 
of children, all cells will be filled except the lower right. Within the limits of sam- 
pling error, Keats found that for several types of paired items, the hypothesis was 
confirmed. One such type is represented by the pair: 

(Concrete) 4 + 5 = - 5 = Answer 

(Formal) B + 5 = - 5 = Answer 

It may well be that this formal -concrete dimension is an important factor in scientific 
creativity. But the difficulty, as I see it, in trying to make use of this dimension in 
a study involving widely different age groups is that an item which might be classified 
as formal for a young group might in the nature of the case become a concrete item 
for a more advanced group. That is, the B + 5 = - 5 = ? item is probably formal 

enough for a fifth -grade group, but for a tenth -grade group that had learned algebra, 
it might be more in the nature cf a concrete item. 

As another example of the problem, I should like to cite Mr. Guilford's Plot 
Titles test. This test, as you know, has a high loading on the factor of originality 
when used with adult subjects. I suggest that if it were given in precisely die same 
form to a group of pre -adolescents, the loading on originality would approach zero 
because its variance would become practically nonexistent. Yet it seems to me that 
we ought to be able to devise a test which would (a) show a high loading on originality 
for pre -adolescents and (b) have a high correlation with the Plot Titles test when taken 
by the same subjects as adults. 

The last problem I shall discuss in connection with the proposed overlapping pre- 
diction study is the one that we always come around to in the end - - namely, the prob- 
lem of developing criterion measures, and I suggest that this problem ought to be 
soived as well as possible at the beginning rather than at the end. As a matter of fact, 
the outline of the study I have described requires that criterion measures shall be se- 
cured at every stage of the procedure concurrently with the predictive measures . 

* 

I think we are too prone to think only in terms of ultimate criteria of achieve- 
ment which are a long way off in the future and of which we are never likely to get any 
adequate measures anyway. Conversely we tend to neglect the problem of developing 
intermediate criteria which eventually we ought to be able to measure with a satisfac- 
tory degree of accuracy . There is no reason to think that we have to wait until a man 
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is fifty years old, or twenty -five,’ or fifteen, or even twelve before we can observe 
whether he acts creatively. What we need. to do is to clarify in behavioral terms just 
what we mean by creative behavior in science as well is in other fields at all levels 
of development and then set up situations in which we can observe it and evaluate it 
under controlled conditions . 

This is of course easier said than done. But it seems to me that if enough people 
bent their minds to the problem we might get some useful inspirations for solving it. 

For instance, there must be innumerable possibilities for setting up situations ii the 
ordinary high school or college laboratory which would give a student a chance to dem- 
onstrate creative power. This is essentially what Brandwein did for his group. One 
might .bring the student into the laboratory for a full day and have him work on a prob- 
lem that was presented without too much structuring. (’’Here are certain materials 
and some equipment. What kinds of questions o 2 cur to you? See if you can find the 
answers to some of them.”) With graduate students something might be done to make 
the thesis more of a test of creative power. And the oral examination might well be 
replaced by a performance test that would give the student an opportunity to show not 
only his knowledge but whether he could use it creatively. Such a test could be fairly 
readily validated by giving it to experienced scientists, some of whom are known to 
be creative and some of whom have the reputation of being merely good routine workers. 
Theses might be validated in a similar manner. The notion I am trying to convey is 
that if we can get solid criterion measures of creative performance at the graduate 
level, it wil probably be unnecessary for us to go beyond this point in order to deter- 
mine who is most likely to become a creative scientist. . 

There are, of course, many other problems connected with the proposed over- 
lapping prediction study, not the least of which is that of devising dependable tests of 

personality factors. But I have to stop somewhere and this seems a good place to do 
it. 

Q--Can you give us a couple of reasons why you think that your comprehensive 
study should work? I see it only as doing more of what has already been dene. I 
don’t see any new approach in what you’re going to do. You are going to get more sub- 
jects, your experimental design may be a little different in terms of a combination of 
things, but why in the world should it work? Why should you at the end of a two million 

dollar study have any better notion as to what creativity is or be better able to select 
for creativity? 

S--I thought I had made it plain that you've got to know what creativity is before 
you can even start the two million dollar study. 

Right. Therefore, I wonder why you didn't spend your talk telling us what 
creativity is. It seems to me what you've done is to suggest a mammoth study which 
replicates every study that has been done, using a larger population, a larger number 
of tests, and so on and so forth. Why do you expect to come up with anything? 

S— All I’m suggesting is that unless something like this is done, after we have 
made up our minds what creativity is, we shall never know how creativity develops. 
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We've got to know what all the factors are that produce it — the training factors, the 
environmental factors, the biological factors, and so forth. I think of creativity as 
something that develops in children as they grow. 

Q--Yes, but the gimmick in your statement is "after we know what creativity is. ” 
How about spending your two million dollars finding out what creativity is? 

C--You could not buy it at that price. 

C— You could sell it for more though. 

S--I think that if you started trying to develop criteria of creativity at a succes- 
sion of points in the line of human development you eventually would arrive at what 
you meant by creativity. 

Q--Do you have any hunches as to different ones that have been used? I am argu -- 
ing against an undertone of this paper, and a couple of others, that whenever we get 
into conferences of this sort on the problem of creativity, someone always says that 
we really don’t know from nothing about this problem, so we’ve got to start from scratch. 
There haven’t been two thousand years of good thinking about this problem that we 
can't build upon. For example, let’s take our criterion measures. There are grades, 
judgments of others, and so on. And we have psychological criterion measures in 
terms of what the creative personality would be, etc. Now, what is wrong with start- 
ing with something like that and building from there? Every criterion measure, if the 
investigator is going to keep his own value system out of it, is only going to be a social 
criterion measure. And this we know from past experience. We can jump in and take 
a variety of criterion measures right at the very beginning and work with those and try 
to figure out what creativity is in terms of the criterion. If you do a job analysis of 
what gets involved in good grades ... , 

S--I wouldn't use grades as a criterion. 

C— All right, then let’s proceed farther on our judgments of others. 

S--What I would prefer would be some kind of performance criterion. 

C--Yes, but even if you have a performance criterion, you are always going to 
have someone making a judgment of the performance. You’re always going to have the 
variable of judgment. 

S--Of course. I don’t see how you’re ever going to get away from judgment. 

C- -Neither do I. Therefore, I say you’re not going to get away from these things. 
You insert the criterion problem when we know what all the aspects of the criterion 
problem are. We don’t have to delay our research in terms of the criterion problem. 

We can take any series of judgments as a criterion. 



is fifty years old, or twenty-five, or fifteen, or even twelve before we can observe 
whether he acts creatively. What we need to do is to clarify in behavioral terms just 
what we mean by creative behavior in science as well as in other fields at all levels 
of development and then set up situations in which «we can observe it and evaluate it 
under controlled conditions . 

This is of course easier said than done. But it seems to me that if enough people 
bent their minds to the problem we might get some useful inspirations for solving it. 
box instance, there must be innumerable possibilities for setting up situations in the 
ordinary high school or college laboratory which would give a student a chance to dem- 
onstrate creative power. This is essentially what Brandwein did for his group. One 
might. bring the student into the laboratory for a full day and have him work on a prob- 
lem that was presented without too much structuring. (’’Here are certain materials 
and some equipment. What kinds of questions occur to you? See if you can find the 
answers to some of them.”) With graduate students something might be done to make 
the thesis more of a test of creative power. And the oral examination might well be 
replaced by a performance test that would give the student an opportunity to show not 
only his knowledge but whether he could use it creatively. Such a test could be fairly 
readily validated by giving it to experienced scientists, some of whom are known to 
be creative and some of whom have the reputation of being merely good routine workers. 
Theses might be validated in a similar manner. The notion I am trying to convey is 
that if we can get solid criterion measures of creative performance at the graduate 
level, it wil probably be unnecessary for us to go beyond this point in order to deter- 
mine who is most likely to become a creative scientist. 

There are, of course, many other problems connected with the proposed over- 
lapping prediction study, not the least of which i? that of devising dependable tests of 
personality factors. But I have to stop somewhere and this seems a good place to do 
it. 

5.' "Can you give us a couple of reasons why you think that your comprehensive 
study should work? I see it only as doing more of what has already been done. I 
don't see any new approach in what you’re going to do. You are going to get more sub- 
jects, your experimental design may be a little different in terms of a combination of 
things, but why in the world should it work? Why should you at the end of a two million 
dollar study have any better notion as to what creativity is or be better able to select 
for creativity? 

S--I thought I had made it plain that you’re got to know what creativity is before 
you can even start the two million dollar study. 

g- -Right. Therefore, I wonder why you didn’t spend your talk telling us what 
creativity is. It seems to me what you've done is to suggest a mammoth study which 
replicates every study that has been done, using a larger population, a larger number 
of tests, and so on and sc forth. Why do you expect to come up with anything? 

S— All I'm suggesting is that unless something like this is done, after we have 
made up our minds what creativity is, we shall never know how creativity develops. 





C--In actuality the judgmental problem involved in setting up experimental cri- 
teria is nothing different from what is involved in correcting Guilford type tests. And 
I think that we are willing to allow that such methods are admissable since they pro- 
vide a way of getting novel responses, etc. Now, these experimental mockups that 
Dyer was talking about would involve a similar type approach. I think he has shown 
us a big pie and I myself don’t feel qualified to evaluate the total pie, but I think there 
are some juicy morsels in it. 

S--I don't think you can evaluate it at this stage. 

2" "Well, wait a minute. I’m going to speak out more in favor of this than that. 

I think it will give you a developmental picture that nobody has now . 

2" -A developmental picture of what? That’s my question. 

C— Of abilities and criterion measures. 

2" "You can see what happens to these measures. You can see, for example, 
what is happening to children in grades 3, 4 , 5, 6, since I assume you will be giving 
the same tests. 

S--Yes, or with slight changes in them. One of the problems is whether the 
same tests at successive levels measure the same things. 

2" "Yes, changes in the measures that attempt to keep the things measured set. 

I'm arguing, you know, that this is not going to be easy, and I agree with Henry. But, 
on the other hand, I don't think that anywhere in the literature (at least we couldn't 
find it) was there any handling of the development problem whatever. 

S--I couldn’t find anything either. 

C— Now, the ways you can go about the development problem are first, the Terrnan 
way, in which you have to live a long time and have lots of money and hope that the var- 
iables that you were interested in in 1921 are the ones you are interested in in 1955; or 
you can take the way of the linked study which you propose here, or the type of thing 
Strodtbeck did. What Strodtbeck did was to study people across generations. He used 
grandfather, father, and son type matching in a somewhat similar design. You can 
pull things out of these linked studies that you can’t get any other way. 

"But the point is that you're going to pull out what you're going to put into them, 
or aspects of it. And I think this becomes a really critical problem. My feeling from 
the presentation was that what was going to be put in was the same old stuff. It would 
stop at the point where something new might be put in. That is, you were going to put 
into this pie all the things that have been put into it in the past. What were the new 
things that were going into this pie? What developmental aspects of real importance 
would it get at? I think it was no accident that you stopped at the personality thing. 

S— What do you think are some of the new things? 
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C— Well, I think now if you start writing the last six pages of that paper we'll 
get them. 

Sr- -It won’t be six pages . It would be a book. 

C--Well, that’s what we need. I think there has been sufficient data collected, 
and although it would be fascinating to know what happened developmental^ to those 
series of abilities, what good would the information do us? I’ll even go further to 
state my bias that those of us in the personality field could even predict the abilities 
from personality data and that if we spent more time thinking about the "what" in that 
area, we could do an even better job of predicting abilities. I will present some of 
them more specifically tomorrow. 

S--One of the problems is that all the personality studies that I have looked into 
are restricted to studies of the- interrelationships in small closed groups. The studies 
don’t tie together. There is no Way to relating what has been found with 20-25 year 
olds to what hag been found with- 10-15 year olds, and so on down the line. I don’t 
know how to tie them together. . 

C — I just wonder whether what is needed is not so much a change in your design 
but simply an extension of it. You cut off chronologically right where' l am beginning 
to get interested. An upward extension doesn’t require any particular change in the 
design; it simply adds another million dollars. 

C— I figured that you would need' 2 million 380 thousand dollars. If you extend 
it into five years in industry, it will cost four million. * 



C--I’d like to introduce an idea at this point that is related to what has been 
said. It seems td*fne that we have talked a good deal this afternoon rabout some tests 
on some abstractive aspects of creative work --for instance, the fnaking of Plot 
Titles . The assumption has been that a fresh plot title or the capacity to make many 
of these things is an indication of creative ability. I think that the* basic assumption 
is that a part will somehow indicate how the whole functions . I doubt very much if 
this is going to be a satisfactory approach . The producing of abstracted aspects is a 
little bit unrealistic. For instance, a creative mind, as I have observed it, is bored 
by the kind of thing that is measured in Dr. Guilford's test. Now that is not anything 
against the test, but it is a criticism of our expectations in using this test - -the ex- 
pectation that We are going to discover creativity by giving a test that would bore a 
creative mind. Such a test does not involve enough complexities. It oversimplifies 
in a disastrous way. 



C--I think I would disagree with you on the boredom, but would agree with you 
that it does not distinguish between the creative and non-creative. 
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C--A state of boredom is irrelevant; perhaps boredom is simply evidence of the 
limitation. 

(J--What kind of people are bored with the test? 

C - -People who are interested in writing the stories that would go with the plot 
title -- who would give the plot title only as minor aspect of the story, the cherry on 
top of the bianc mange. 

Q ~ Is there not perhaps an important distinciton here between the artistic creative 
and the scientific creative? 



C-No. 

C--I think. I’m talking about artistic creativity simply because the test I was taking 
as an example involves or implies artistic creativity. 

C ~ We took a group of Nobel prize winners, and starred men of science, people 
who are really outstanding in chemistry and mathematics, and I do assure you that they 
were not bored by these tests. In fact, they were almost child- like in their approach to 
many of them. 

C- -Yes, but that perhaps is evidence that the really first-class mind is delighted 
by almost anything. (Laughter! ) But I wouldn’t believe it. I think that probably they 
were interested in the testing because it was something fresh, because they thought.it 
might lead somewhere, because they would be bound to be interested in creativity, and 
because their expectations were high. 

C--I wouldn’t doubt any of these possibilities, but my main point is that the sub- 
jects were interested when they took the tests, and they did about as well as we would 
expect them to. 

C--Wilson is going to give some tests on gullibility. I think he inferred that the 
people who scored high on gullibility would be the less creative. I should think it would 
be the other way around-- the real creative person is the one who was gullible enough a 
few years ago to have accepted the notion that you can make clothes out of coal and metal 
out of sea water. 

C- -There is no creative goal there in the Hot Titles . I wonder what you would 
say about that, Mr. Guilford. 

Q--The creative goal? 

C- -There is no creative g oal there: actually, the creative goal is absent. 

C— Well, if you mean that the person in taking the test has set the goal for him- 
self -- no: the goal was set by the examiner. 
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C--And the goal was to produce plot titles, I guess. 

C— Yes, the goal was to produce a good score. The subject is told how to do it. 

Q ~ Does that constitute a creative goal? I think r »ct. 

Q— Well, in approaching that goal does his performance in the test indicate, as 
compared to someone else, that he has more originality or that he has more fluency of 
ideas? That ! s the point. 

Q--It certainly measures fluency of ideas and ability to create something doesn’t 
it? But does that lead us to creativity? 

Q -- You think that this approach is an oversimplification? 

Q— Well, I think you can simplify and still get to creativity, but I am not sure 
how you do it. Is there a possibility of setting up one of your tests. Dr. Guilford, so 
that the person could choose what he did with the item? 

C— We have one test of that sort and it was a failure in predicting anything. 

Q>--What did you do exactly? 

C— Well, each item consisted of a peculiar marking of some kind- - a letter or 
the beginning of a word or what not. The only instruction was to do something with 
each item and to do as many items as possible. 

C — I meant some tiling more specific than that. You have some tests in which you 
require evaluation, other tests requiring fluency, and so on. I'm wondering if, for ex- 
ample, you could have an item where the person could either be fluent or evaluative, 
without telling him which. I’m not quite sure how it would work. 

C— No, we don’t have an item like that. 

C— You see what I’m driving at is that you might give the subject a freer choice. 

C— You want to let him show whether he tends naturally to a higher level of crit- 
ical mindedness or a lower level. 

Q — A more open-ended sort of thing, as far as the subject is concerned? 

C--Yes, that’s right, so that you can score it by either of your systems. 

C--That goes against my own inclination to control experimentally the conditions 
under "which he does his work as much as possible. Maybe that open-endedness would 
defeat the purpose of the test. 
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C--The conditions are still controlled, in that the subjects are all exposed to 
the same item. But the thing we want to find out is what this person prefers to do. 

That is, what is the high habit in the habit family hierarchy? 

£ I can predict that the freedom that you would give him would simply intro- 
duce more factors into the test and make it more complex factorially. 

... 2~ That really raises a Question I was about to ask, or Derhaps it answers it, 
would it be possible with your method of testing to create a satisfactory test which 
would end with a work of art, however simple? 

Q--Such a test would almost be the criterion, wouldn't it? 

C” t is certainly factorially complex. 

~ I mecti to say is that no test apparently has ever been invented that 

would yield the whole criterion, yet all the tests sample the criterion. 

C--Yes, they do that, 

-~" At Coo P er Union, in the battery they use for selecting students of architecture 
they have a test that requires the subject to create a drawing. Is that the kind of thing 



C I m sorry. I meant a factorial test, a test involving perceptual factors. 

5. ""By definition it would be factorially complex. 

. ~“ I *\ ave on e result that might throw some light on this question of the control 

and what it does to the factorial composition of the test. Originally, we gave the Plot 
1 les test with no instruction about being clever; it wasn't mentioned at all. Clever - 
ness was spontaneous so far as the examinee was concerned. In a later analysis, we 
changed the instructions to say that cleverness will be evaluated in scoring this test 
or words to that effect. The effect in that group, although we don't know whether the 
group is comparable to the other group in every way, was to lower the fluency score 
and do nothing to the originality score. Probably their increased self-criticism put a 
damper on quantity of responses but did not change the amount of creativity. 

C--You have rattled an old skeleton in the psychological closet - -the old argu- 
ment of whether or not the test should be a replication of the job, or whether or not 
the psychologist should have the freedom to introduce as much abstraction into his 

! e !L aS h »w !? e f ‘ 11 ha f been f0Und that fr a £ men ting the total performance into such 
tests as Plot Titles results in ease of scoring and ease of handling fnr n,iriw«« c 



ui cApciiuientai situations. Then its merits will stand 
relationship. 




3 Dr, Dyer suggested, in terms 
or fall upon the strength of that 
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C--It really is expressed then in terms of the analogy that you have to kill a 
man before you can anatomize him. 

C--Well, we hope the subject survives whatever we have to do to him. 

C--I have an incidental observation on whether creative people are bored by the 
Plot Titles test. I think, in general, practically any decent human being hates any 
psychological test. And creative people hate tests of originality - -but do well on them. 
I have observed this a number of times. The subject will object to the test, saying, 

"I suppose you are trying to test my cleverness. " But he will do well on it in spite of 
thinking that it is a very poor device . 

C--A research director has proposed to me that if I find any people who refuse 
to take a psychological test, he would like to hire them. (Laughter. ) 

C--A11 this may mean is that we will have to design originality tests especially 
for the kind of group we are dealing with, its motivation, its level, and so on. 

C- -That's a very good suggestion. 

C--You can have challenging originality tests. 

C--You could have them for the kinds of people who are bored with the present 

ones. 



C--Our general feeling is, I think, that when we are dealing with scientists we 
would like to give them problems as close to the science they are working with as 
possible. I think that is the main point I was trying to raise before. And I think it 
comes close to this point. If I am going to test an artist or a poet I try to give him a 
problem as close to his main interest as possible. This makes for a very complex 
test. It is not only factorially complex; but it is also complex in terms of the tremen- 
dous variety of subtests that have to be developed. 

C--A terrific amount of self-selection goes into any creative act. I mean, such 
an act involves repudiation of a lot of alternatives and a selection of what to do. Of 
course, when you give the same test to everybody, you know that you can't permit the 
self selection sort of thing to manifest itself. Nevertheless, even though the examiner, 
not the subjects, has selected the medium, the assumption is, and the results seem to 
indicate, that the subjects do perform in an original manner. They manifest their orig- 
inality even though they are not free to choose their own medium and would have chosen 
a different one if they had been given a chance to do so. 

C--Well, I think my problem could be handled by keeping the two things separate. 

I can see how it would introduce error and lead to factorial complexity to allow the sub- 
ject to select alternative media. But then you can take care of the situation by giving 
another test independently which will give you a measure of motivation. The one thing 
that you can't do - -at least we can't do it - -is to get a measure of motivation by any 
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kind of structured test that relates to performance. We'd have to leave it up to the 
subject, because what he freely chooses to do when you give him a choice is the best 
indication of what he wants to do. It is a much better indication than what he says he 
wants to do as far as performance is concerned. But then, you can get that measure 

independently, and then get your measure of ability, and then run your multiple re- 
gression. 

2- Since we are dwelling on Plot Titles, I think that Dr. Dyer has raised an 
issue that is a challenge to us and that we have left unanswered. That is his question 
on how we can develop a test for 6- or 8 -year -olds that will measure a factor complex 
similar to that found when older subjects are tested. I was wondering whether or not 
a plot can be reproduced in terms of a picture and whether vou could not relate the re - 
suits of such a test to what the man will give you later in life. Certainly the mission- 
ary in the boiling pot with the princess on one side could be reproduced in colored pic - 
tures. I don't know if some of the other plots are more complexly determined, 

2—1 may be just expressing my ignorance, but is there any evidence that. we are 
anywhere near being able to maintain factorial constancy over different levels? 

S--I don't know how we would ever determine for sure that we had. That's what 
bothers me. 

C--This design would have to tolerate a tremendous amount of error. 

S--This is the problem as I see it: How can we know whether we are measuring 
the same thing at different levels? 

C- -There are some answers being sought for that. Cattell has developed what 
he calls the Junior Personality Quiz, and he attempted to determine the factor compos- 
ition of that thing. But I think that similar attempts in the cognitive area might not be 
as difficult as for some of the personality tests. 

S-The problem is not just a matter of dream^ig up the tests; it also has to do 
with what kind of analysis you can perform and what kind of data you are to get out of 
the tests to answer the question, "Are these tests actually measuring the same thing 
over different age levels?" 

C--I wonder if you really need factorial consistancy through here to solve the 
real problems that we are concerned with. Are you really trying to predict at age 
ten who's going to be the senior scientist forty years later? You're concerned rather 
with opening up for the ten year old the possibilities which permit him to go ahead, 
maybe for the next three or four years. If so, you are solving a different set of prob- 
lems. Your problems change with chronological age, with progress through the edu- 
cational system, and you really don't need factorial constancy here to solve the real 
problems that we are concerned with. 

C--I don't think that you want factorial constancy as much as you want a multidi- 
mensional picture of abilities to relate to a criterion at a given age level. 
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C--You want a constancy of concepts . 

S--But I do think that you want some kind of factorial constancy to get at th'e prob- 
lem of what kind of training the children ought to receive . Educators repeatedly ask: 
"What is the natural development of the child?" We ought to be in a position to tell 
them what sequence of things the children should be taught. We can’t answer that 
question now. We don’t know whether natural development goes this way or that way 
because we don’t now know what the effects of different kinds of training may be on . 
various aspects of development. 

C--I think that there is a more embarassing question. We don’t even know what 
to tell an administrator about the factors which should go into this pot of concepts. I 
think that this is the critical problem -- not the empirical factors, which will turn up 
in new tects . We could spend to the end of our days developing all kinds of tests that 
will yield new factors. But what are the concepts that describe the individual as he 
is going to develop? They should govern the development of our tests . This is the 
approach we should take rather than to develop the tests, see the factors, and then 
try to do longitudinal studies with them. I think that when we give these tests of abili- 
ties, we’re doing not much more than a refined job analysis of what the significant 
others in the environment will approve of. These significant others are already work- 
ing in terms of known criteria. Our role as psychologists is to open up additional areas 
and concepts which they have overlooked and which probably are going to be the critical 
ones. 



C--It seems to me that this design would provide for that; that is, this would 
turn up some new concepts and new intellectual qualities which we could emphasize 
more than we do now in education. 

C--I don’t see how it would, because not enough time is allowed in the design 
for thinking through the theoretical problems with regard to the sociological factors, 
the training factors, and the educational factors as ‘well as the theoretical cognitive 
and personality factors in the individual . We know already that all the work in the cog- 
nitive, intellectual area has yielded correlations of the order of . 50 to . 60, if we are 
really optimistic about it. But is that what we are really hitting for? We’re hitting for 
something much better than th :, and we need other material. 

Q--I wonder, though, if you intentionally wanted to exclude motivation. 

C--No. Thurstone suggests that the mo3t critical event in the whole creative 
process occurs prior to the moment of insight, and he proposes a test for measuring 
what happens then. Thus far, if I am able to understand the. tests that have been ex- 
plained, I don’t think that we have had a recommendation for one test that would be use- 
ful at that critical point. 

C--I can sympathize with Dr. Stein’s concern about having missed something 
even though we have a large number of concepts already to work with . Nobody has 
mentioned here so far the concept of incubation. I think that that is a very important 
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things and that it has been neglected. We haven’t found any way yet to work with it, 
but it is very important. Incubation occurs just prior to the moment of insight, and 
your statement reminded me of it. 

* 

C--If you come right down to it, there has been so little done on changing factor 
patterns that I think that all of us would have to revert to Anastasi’s study. There 
would be very little else in the literature to show the effects of specific experience 
on changing factor structures. 

C--This is because factor analysts are not experimentalists. We tried it once, 
but we got confused trying to interpret the results . Cattell actually suggests doing 
this sort of tiling, but he never did it. That is, he suggested that we ought to see if 
we can change factorial composition by introducing specific experiences of success 
and failure. 

C--Well, he has a large amount of data on identical twins reared together and 
reared apart, and a study of their personality differences. But it is not in twin studies 
where your experimental variable is to be found. There is no experimental variable 
there. 

Q- -Aren’t there some fancy sampling problems here if you want to have any kind 
of comparability among groups? It seems to me if you go into more and advanced 
schooling, you will be running into a self- selection process, so that when you get into 
graduate school you’ll have only a few who are comparable to the third graders. 

* » 

Sr -Yes, that’s true. 

C--If you don’t care, you could throw out those who didn’t go to college eventu- 
ally. (Laughter. ) 

C- -There is one other developmental problem here. I think that it could be said 
that a single person’s creativity changes with time, and that there are some kinds of 
life circumstances which would lead to an increase in creativity, and others which 
would lead to a decrease in it. But you have to think somehow in terms of a threshold 
for the emergence of original acts with environmental circumstances being part of 
what determines them. 

C--That highlights a very important fact: the creative process does not occur 
only because an individual is constituted in a certain way; at least as important is the 
fact that he behaves in a certain way. Primarily it is his behavior rather than consti- 
tution that we should be concerned with. 

C--Yes, it is, of course, but we have adopted a curiously static approach to what 
creativity is in all of what we have spoken of. We’ve accepted all the contrasts that 
science has always made, all the establishment of constancies, which of course is 
necessary in some respects. But still with regard to creativity, I think that probably 
you have to get beyond constancies in order to make much progress in getting insight 
into the process itself. 
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C- -Creativity is an act in iteslf. 

£--We are trying to study creators, are we not, rather than their act of creation? 

Q- -How can you study them except at the moment when they are acting? The^ are 
not creators all the time. Isn't that true? 

C--I think that one of the assumptions we are making is that the things that enable 
peoplelo be creative are things that they exhibit under conditions other than those in the 
creative act itself. 

C--Yes, there is a disposition to produce original acts, I think -- a generalized 
disposition which is fairly enduring and can be called a structure as much as anything 
is a structure in personality. And beyond that you have to think about how the changing 
environment allows that disposition to emerge. 

C— This, however, is an assumption, and we ought to be very clear that it is an 
assumption. 

C- -We've talked here about creativity largely in terms of abilities. Now we are 
getting" around to the question of motivation. I’d like to suggest that there is pretty 
clear evidence that creativity should be thought of, at least for the most part, as a 
matter of motivation. Abilities are relatively constant. We know very well that crea- 
tivity pre-exists within the individual. We know that the same biological stock is crea- 
tive in one effort and uncreative in another . We know that one man may be creative in 
a certain restricted area and not creative in an immediate adjacent area. Now, all of 
this impresses on me a need for thinking in terms of the dynamic motivational factors. 

I mean that creativity is an act of connation in the private sense. It requires the appara- 
tus of ability, but the apparatus of ability alone will not take us very tar, I think, in 
understanding the final act. 

q— I’ m glad that you raised the motivational component because I was just getting 
aroundto dropping this on the group. How many among you are now willing to consider 
creativity on the level of a drive? How many of you are willing to talk about a creativity 
drive? Are you willing to accord it that recognition? We have approached other complex 
phenomena and accorded them the status of drives - -for example, anxiety and achieve- 
ment. There is something that is measured on the Rorschach called fulfillment which I 
think is pretty close kind to creativity. Is it worthwhile to tackle this thing from a drive 
level? 

C--I think that it would be dangerous to talk about a unitary motive. On the other 
“hand, to fail to see that creativity is related to the total dynamic structure seems to be 
an extremely serious oversight. I think that creativity may be related to certain defense 
mechanisms, for example. Now, is it a drive or some complex evolution of drives? 

C- -I think that you have a point there. I think that it is a derivative, but, of 
course^ since psychologists always get impatient trying to get at the ground fromwhich 
the derivative comes, they strike directly at the concept and with varying degrees of 
success. I think that probably creativity is a derivative of curiosity, self-assertion, 
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desire for recognition, and a lot of other things that some people who define things in 
terms of the McDougallian concept of drives, are trying to operate with. But on the 
other hand, the creative drive seems to manifest itself within persons in what the 
factor analyst calls the unique factorial structure of the individual. In other words, 
a man with a high drive for recognition may express his drive by becoming a concert 
pianist. I think that there is no other area like creativity where a man makes a de- 
cision early in the game and channelizes his energies into a rather narrow band. And 
that is why, perhaps, the greatest success in prediction is going to come from a test 
format developed in specific areas. This point was brought up earlier. I d be willing 
to approach it either way. I’d be willing to approach creativity directly as a drive, or 
to search for the lower order array of elements that mav enter into it and then to 
identify it, possibly, in the second order level of the factor structure. 

C- -Before we close this session I would just like to make one comment. Dr. 
Dyer commented on his belief that science should be studied in this general complex 
of areas along with other fields. I would like it crystal clear that the National Science 
Foundation shares this view wholeheartedly. We are supporting this conference with 
a specific title to be sure, but I am reminded by some of this discussion of a comment 
made in our office a few weeks ago by an official in one of the larger private founda- 
tions, to the effect that scientists nowadays seem to be the ones who are much con- 
cerned about the humanities and the social sciences and what is going to be done with 
them over the next few years. So many people have jumped on the band wagon of ob- 
taining more and more scientists that the scientists themselves are having to take up 
the cudgel for the other fields . I simply wanted to say, therefore, that certainly from 
our point of view, studies that would give us information about science only would be 
, of little utility . Now, this is not to say that we can support such studies in across- 
the-board fields because there are certain limitations on us for practical reasons, but 
we are interested in the overall gamut of studies rather than just studies pointing to 
science alone. 
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AN INTERIM REPORT ON CREATIVITY RESEARCH 1 

Maury H. Chorness 2 
Personnel Research Laboratory 
Air Force Personnel and Training Research Center 



S- -In the Personnel Research Laboratory of the Air Force Personnel and Training 
Research Center, we have been conducting a limited research program on creative abili- 
ties. This research has been prompted by our desire to identify human talents of a high- 
level nature which do not seem to be measured at present by selection tests employed by 
the Air Force. In terms of operational needs, rich ideation may be considered to be at 
a premium in the Air Force at large, for a very extensive incentives program is in ex- 
istence, replete with elaborate administrative machinery for processing individual pro- 
posals. Originality, with its fluency and spontaneity counterparts, is also gaining great- 
er prominence as an attribute to be considered in programs of officer assessment and 
instructor selection. 

Our desire to conduct studies in creativity appropriate to the Air Force has been 
prompted by the early work which has been accomplished by Guilford and his associates 
in their measurement of certain dimensions of creative thought . An examination of the 
factor structure, arrived at by the Guilford group, revealed that several new dimensions 
were being measured by several short, quickly administered tests. We were further en- 
couraged when it was noted that these new dimensions were quite independent of those 
identified by our Airman Classification Battery, which is the backbone for selection and 
classification in the Air Force . Since the Air Force reference battery revealed little 
or no overlap with the factor structure of the creativity battery, we developed a plan for 
the application of the new tests in a variety of situations wherein such unique variances 
might be found to have predictive value. In some of our studies, these tests have been 
supplemented by a personal data questionnaire, which is principally biographical in 
nature and slanted expressly toward measuring out aspects of the person's background 
conceivably related to the development of creative talent. 

In the Air Force we maintain a rather close contact with the developments that 
have gone on at the University of Southern California. Many of the members of this 
organization were in the original Air* Force program to set up the selection batteries 
for pilot bombardier, navigator, etc. Dr. Guilford, as you know, fathered that pro- 
gram, and I would like to say that the tests that were developed in conjunction with that 

*The opinions or conclusions contained in this report are those of the author . 

They are not to be construed as reflecting the views or endorsements of the 
Department of the Air Force . 

2 The program of research described in this article is that undertaken by the 
Prediction Research Branch, Personnel Research Laboratory. Grateful acknowl- 
edgment is made to Dr. RobertM. W. Travers, Chief of the Prediction Research 
Branch, for the suggestions in the writing of this manuscript. The writer is 
now engaged in curriculum analysis and research with the Officer Military 
Schools, USAF, Lackland Air Force Base. 



program still stand today and defy students of motivation to produce predictors of elim- 
ination in flying better than the battery itself does. And to a great extent, those early 
developments have justified the later developments in the Air Force. Psychologists 
endeared themselves to the Air Force by coming along at a moment when there was a 
desperate need for selection instruments. 

Incidentally some of the work we are doing ties in with a certain point of view 
that was voiced over and over again in the previous days. That is to say, the relation- 
ship between the kind of segmented outbursts of creativity that we get in the fluency tests 
and more integrated aspects of creativity, and we have attempted to devote our attention 
to criterion areas that represent the integrated type of behavior that may be subsumed 
under creativity. And we thought that instructor performance, where a man gets on his 
feet, and has to present material to a class, represented this kind of integrated behavior. 

As a brief review of our program, the following studies are briefly described. 

~a. Instructor selection has been a perennial problem in the Air Force. There- 
fore, in one of our projects we have developed rating scales for certain definable as- 
pects of instructor performance, and have attempted to determine their predictability 
from tests of creativity. 

b. -We have taken up the hypothesis advanced by Thurstone to the effect that the 
creative person may possess ’’good rapport with his prefocal thoughts. ” This study has 
much in common with experiments in concept formation. 

c. In conjunction with the Air Force Incentives Program, we have spent consider- 
able effort in identifying and testing awarded and non -awarded civilian employees. This 
is a controlled study, with matching of groups on certain relevant variables. 

d. The Creative Engineering Laboratory at M.I.T. is as nearly unique a criterion 
situation as one could hope for in creativity research We are engaging in a joint study 
with Professor John E. Arnold, director of that laboratory. In this study, we are attempt- 
ing to predict ratable aspects of creative design. 

e. At Lackland Air Force Base, a lively interest has been generated in creativity 
among the people responsible for the evaluation of training curricula. We have been in- 
strumental in setting up a research design which is being pursued at present in the Of- 
ficer Military Schools. As a unique aspect of this study, Dr. Borg and Dr. Zaccaria, 

of the Officer Military Schools, have been able to bring together two prominent develop- 
ments in the general area of creativity which thus far seem to have gone their separate 
ways. These are the test developments of Guilford and the instructional procedures of 
Osborn, who founded' the Educational Creativity Foundation. I plan to review, briefly, 
the techniques employed Jry Osborn in the event some of you are unfamiliar with them, 
since the evaluation studies conducted at Lackland by the Training Analysis and Develop- 
ment Unit encompass several of his principles. 

f. We have also been carrying on considerable discussion in the Prediction Research 
Branch concerning the possibilities of a laboratory approach to the measurement of 
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creativity. I note that the conference agenda is intended to include discussion of an ex- 
perimental approach to creativity, and it will be interesting to see whether Our deliber - 
ations have any coincidence with the thoughts entertained by others at this conference 
with regard to the same topic. 

Finally, I should like to get your reactions to some ideas which lie in the realm 
of personality and social psychology. Certain prominent features from these two areas 
have impressed us greatly in the field studies we have conducted, for we have met highly 
creative individuals on a personal basis, have interacted with their supervisors, and 
have noted differences in group climates which do seem to be associated with the idea- 
tional productivity of groups . 



The Predictability of Creative Expression 
In Instructor Performance 

The Air Force expends considerable effort in preparing personnel for instructor 
positions covering a wide range of subjects. In fact, an instructor school has been in 
existence for some time at Lackland, having been established for the express purpose 
of coping with the large turnover of instructors and for the purpose of indoctrinating 
newly selected student instructors in their future responsibilities. Selection for the 
school has been based mainly upon an intelligence index derived from the Airman 
Classification Battery. 

Personnel in charge of the school were interested in supplementing their selection 
techniques, and agreed to a joint study in which we would administer several of the 
Guilford creativity tests, in addition to collecting observational rating scores from one 
phase of the course. In the phase we were concerned with, each student instructor was 
required to deliver a lecture -discussion on a topic of his own choosing. We developed 
several rating scales through which we intended to assess, by observation, those counter- 
parts of each student's performance in his lecture -discussion which could rightly be con- 
sidered demonstrative of creativity. Accordingly, we developed graphic scales for audi- 
ence interest and reaction, degree of spontaneity, ability to transcend rote material, 
originality of expressions, etc. We did not wish to be content with a course grade or an 
overall supervisory rating, but wished, instead, to structure our criterion performance 
in terms of observations hypothetically related to the traits measured in the creativity 
battery. The students were tested several weeks prior to the delivery of their lecture- 
discussion. In addition, they were notified about the kind of performance they were ex- 
pected to give, with sufficient time being allowed for preparation. In actuality, the di- 
rections were a "watered -down" description of the creativity dimensions, but written 
appropriately for the speaking medium. I am sure that most of you are familiar with 
these dimensions which are considered by Guilford to be expressive of associational 
fluency , ideational fluency , originality, spontaneous and adaptive flexibility, redefinition, 
and sensitivity to problems . While we tested and observed a greater number of students, 
our sample was finally reduced to 50 because of eliminations and wash-backs. So the 
findings I have to report to you comprise result from the restricted group which com- 
pleted the entire course of training in the Instructor School. 



Since we desired to build up reliability of our ratings by composites, we conducted 
a factor analysis of nine rating-scale intercorrelations . Four factors were extracted, 
with three of them being identified as audience reactivity , native id eation, and problem 
sensitivity . The rating scales defining these factors were combined into their respec- 
tive composites. The ratings on global originality and spontaneity did not load signifi- 
cantly on any of our dimensions. However, we retained them, for we were cur? ' ' s to 
see how they would relate to similar components expressed in the test media. We $lso 
derived composite scores on the creativity tests, clustering them according to factors. 
The multiple correlations between the factor composites from the creativity battery and 
the criterion scores are shown in the tables at the end of this report. Table 1 represents 
the betas and multiple correlations when we included a measure of general intelligence 
derived from the Airman Classification Battery. Table 2 represents similar relation- 
ships when the intelligence score was excluded from computation of the beta weights. 

If we are to judge from the small reduction in some of our multiple correlations 
when intelligence was dropped out, it would appear that the creativity tests can carry 
the burden of prediction. It is interesting to note that our single best predictor is a 
test of controlled associations, in which the subject must think of words which aro sim* 
ilar in meaning to the stimulus word. We employed this test as a single measure of 
Guilford’s associational fluency factor. At first blush, it was disappointing to note that 
file ideational fluency factor was negatively related to our criteria. However, after a 
moment’s reflection upon the scores making up this factor, we recalled that it is com- 
prised of low quality scores. Since clever and imaginative lecture -discussions received 
the higher ratings, there seems to be some justification in discovering such a negative 
relationship. It is also interesting to note that our factor composite predicts the school 

phase grade much better than the intelligence index which is the sole means of selection 
at present. 

The training people at Lackland are quite willing to adapt the tests which show 
promise as predictors, and it is quite possible that improved predictions may occur if 
additional studies should be conducted in which a broader range of talent, namely the 
eliminess, is also included. In future validation studies, it will be necessary to increase 
the range of observations of the criterion performance. For example, we did not include 
a sufficient number of rating scales to define a spontaneity factor, which ought to be 
demonstrative of fluency . It is quite possible that tests related to the communication 
process may turn out to be valid predictors of instructor performance. 

The other day it was mentioned that some of these tests do not correlate, that the 
originality variability doesn't correlate with grade scores in universities. We found 
that the originality composites did correlate with the phase grade score that the instruc- 
tor gives each student on this particular phase. I’ve just named the dimensions them- 
selves, the factor dimensions, which are represented by composites. It should be men- 
tioned, however, that the associational fluency dimension is only represented by one test 
and that's the Controlled Association test mentioned earlier, and we get fairly decent 
data with that test against all of the criterion areas. However, it is a simple test. Do 
you suppose it's the time element that gives it a better relationship? 

C— No, because some of our other tests required greater time. 
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C- -Saunders also scored his responses differently than you did. You scored your 
total number of responses whereas Saunders gave a point for each common response 
and gave no credit for very remote responses . 

C - -But these two scores would probably be quite highly related. 

C~ There's no reason to expect that the intelligence index should contain spontane- 
ity components. I think it’s justifiable to look for that in the creative battery. 

Q~ And the phase grade was just a grade on one course? 

S- -Yes, it was a grade on that course you see, in which we were conducting our 
studies; we didn’t want to get an overall grade, necessarily, because it included many 
acliievement components. 

Q--Did you ever do any sociological studies? Attempting to correlate background 
variables with success — as a teacher? 

S--We have a life -experience inventory which was not built by the time of the 
study, and we’re using it on some other studies. It has to do with that exactly. We’ve 
attempted to structure some biographical items that will refer directly to parent -child 
relationships, teacher -student relationships, etc. 

C--I didn’t really mean that. I mean just what occupation was he in; or what was 
his father in? One would suspect that if he had been a teacher or his father had been a 
teacher, that he’d be a better bet. Usually the sociological variables carry a lot more 
power than any tests. 

S--I can tell you that the people are chosen from diverse backgrounds because 
they sometimes have to dig deeply to get instructors to take care of the turnover . 

C — My point is that your interest is only in prediction and I assume that it is 
not — that you really want to understand the problem. But if your interest is only in 
selection - -in prediction -- if you made up an index based on whether or not the person 
was in an occupation that required language or his mother had ever been, or his father 
had ever been, this would be a better predictor — I’d be willing to bet — than all your 
tests put together. 

S--I think you have a point there. Because I was right in on the test observations 
and administrations, and where you have a college group getting into school with this 
kind of thing you’re mentioning, the whole level of response goes up and the whole nature 
of the kind of talks they deliver goes right up. Now there's been a push just recently to 
do long range validation studies; you see these people are going to stay with the Air Force 
four years, and they’re going to be right in the Training Command and their retainability 
is good; and so there’s a push now to put a more sensitive battery into the field. Perhaps 
some sociological variables ought to be included in a biographical questionnaire. 

C- -Could you go over once more those tests in this battery? 
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S — Well, for the associational fluency factor., we used a Controlled Associations 
test. For the ideational fluency factor, we used Plot Titles, Unusual Uses, Consequences, 
Brick Uses, and that’s about it. 

-How did you get the composites? 

S- -Intercorrelate the test scores and run a Kuder Richardson #20 on the relia- 
bility for the composite; standardize the scores and lump them again. 

Q--Your weights are determined independently of your criteria? 

S — Yes. We used unit weights; we didn’t bother elaborating the weighting system 

at all. 



2* “You just tried to get the most reliable composites? 

S--Yes. For the originality factor we used the total score from Plot Titles, the 
statistically uncommon score from Quick Response, remote consequences from the 
Consequences test. For the spontaneous flexibility factor I mentioned Unusual Uses 
earlier, I think Unusual Uses also loads on spontaneous flexibility, and I think we de- 
cided to include it there along with the Brick Uses flexibility score. We used a Gestalt 
Transformation test, which is a structured multiple choice test for the Redefinition 
factor and for problem sensitivity, we used Social Institutions; we didn't use the Common 
Situations test for that particular study. 

Another point worth mentioning is that the originality composite did correlate posi- 
tively with the phase grades . 

C— I think this is a very important study apart from the prediction aspects of it, 
the evaluation of the instructors themselves - -a great step forward I think. They em- 
phasize the kind of things that we look for in teachers that inspire students to go into 
science, for example. 

S— I think that there’s a spontaneity dimension working somewhere in there, and 
if we run studies again, it should be supported by several new rating scales, on a boot- 
strap approach - -go along, keep structuring this criterion performance hoping that even 
though you’re fragmenting it by rating scales you’re still assessing an integrated crea- 
tive performance and seeking relationships with predictability tests. There’s no doubt 
that the pie is big and there are personality components in there, and the training school 
is assembling a battery of tests which will include aspects of personality. 

Q--Do you have any chance here to compare these ratings? You indicated turnover 
is a terrific problem; I wondered whether people who show up well in this training course 
for instructors are the ones who stay on. 

S- -They are, indeed. It is interesting to watch the elimination rate. One time you 
get a lot of college trained people, another time there’s a dearth of them. All the time 
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though, you’ve got to supply the Training Command with a flow of instructors who take 
care of the turnover; so the wash-out tends to go up when the caliber is poor and tends 
to go down otherwise. So our study handles people who come in under various conditions. 

Q -Did you study the kind of people who washed out on these tests compared to 
those who stayed in? 

S--No, but we have been talking about setting up a pass-fail split and correlating 
the Guilford tests against it. 

C - -Maybe the ideational fluency factor will correlate more with the scores and 
performance of those who wash out. It's quite possible that they give low level talks 
which would correlate with a low quality score. 

Q- -Did you have a restriction of range on some of the scores? 

S--You mean by virtue of not including the eliminees? I think so, yes. 

Q- -Among those people that washed out, were some of them tested? Did you get 
the criterion information on them? 

S--No, because some of them wash out before you get to that phase. They’re 
subject to wash out at any phase - -there's a counseling system set up and students are 
counseled for a trial period. If they don’t work out, they go back in the pipeline and 
wind up in a tech school somewhere or as tactical instruc )rs . 

Q--What good is this? If you are trying to select in advance, shouldn’t your cri- 
terion be a wash-out one instead of a performance one? 

S--Didn't you bring up the idea that criterion will vary with admission? We're 
very criterion conscious in the Air Force. It hits us in the face everywhere we go. We're 
tired of the gross dichotomizations that conceal the qualitative things that we’ve been 
trained to look for. We try to do something else besides this gross split that has been 
employed in the past for emergency purposes as in the AAF psychomotor program when 
they were developing selection batteries for pilot, navigator so on, there was no cri- 
terion. So a test had to justify itself by predicting the pass -fail split, in addition to de- 
fining a new factor. 

C--We ran reliabilities and in some cases they're extremely high, in some ex- 
tremely low, and that is why we attempted to build up our reliability ratings by cluster 
ratings. We also had another problem of three or four classes going simultaneously and 
having to introduce a large number of raters. There’s a limit to what you can do in an 
operational setting. We wanted to get about two teams of raters and have them do the 
thing all the way through but it was impossible. 

Q— So you have reliable ratings there as well as some very poor ones? 
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S--Yes, for example, I mentioned that originality as a global rating and spontane- 
ity do not fall into any dimension. Those two are not too good in terms of their reliability. 
The cluster ratings are liable to run up into the . 80's, but the single rating elements -- 
their rating reliabilities based upon average interrator reliability --would be down around 
• 35 or .57, or something like that. But I think the spontaneity rating can be supported by 
other ratings ganging up on the same phenomenon, but measuring it a little differently so 
that the rating reliability could be built up that way. 



Applications of Thurstone's Hypotheses of Pre- Focal 
Thinking in Creative Research 

Perhaps some of you will remember that a rather interesting paper was written 
by Thurstone, which he entitled Creative Talent. In that paper, Thurstone suggested a 
variety of approaches to the measurement of creative ability, and many of these ideas 
are being carried out today by various agencies which are engaged in creativity research. 
Now it is only natural, in his pre -occupation with mental abilities, that Thurstone should 
attempt to define differences between the mental activity of highly intelligent people as 
opposed to that type of thinking carried on by the creative person. It also seems to me 
that the search for an operational definition of creative ability which will distinguish it 
from intelligence, as normally measured, is a crucial step in the justification of new 
test formats. Therefore, I should like to repeat Thurstone's point of view, stated in his 
own words. 

"According to one theory of intelligence, the psychological act originates in the 
essentially affective and non -verbalized, non -focal motivations and needs of the individual. 
The development of the psychological act towards final overt expression consists of suc- 
cessive particularizations. Each of these successive steps can be regarded as a choice 
point or bifurcation. If the choice point is in focal consciousness, then the decision is 
subject to rational control and it is fully conscious . If the choice point decision is made 
unconsciously in the particularization of the act, then the choice is determined instinc- 
tively or by intuition, habit, or chance. According to such a theory, a high intelligence 
is indicated by a choice point that becomes focal in consciousness at an early stage of 
the act so that a wide range of possible overt expressions is under conscious control. If 
the act develops under great pressure of motivation or emergency, then it will develop 
to nearly overt definition before it becomes focal in consciousness and the act is then 
likely to be impulsive and relatively less intelligent. It seems plausible that there should 
be individual differences in the ability to be in some kind of rapport with the unconscious 
stages of the act, even before it becomes focal in consciousness. Imaginative people 
might have some kind of rapport with their own unconscious thinking before it becomes 
focally conscious. It might be possible for a person to have the ability for a high degree 
of abstraction, in that his acts are subject to conscious deliberation at their early stages, 
and still not be in rapport with his own unconscious thinking. Such a person would be in- 
telligent and even profound but possibly not creative. " 

As a means of testing this hypothesis experimentally, Thurstone has proposed a 
typical concept formation study. However, it was his intent that the attention be shifted 



from the g6n6r£il process of concept formntion to individual differences in le&rning be 
fore insight. The study of pre -focal or pre -insight learning, of course, is not without 
its counterparts in previous studies. In an earlier study, Thorndike and Rock interpre- 
ted a gradual rise in the learning curve to mean that learning was proceeding without 
awareness. Rees and Israel found they were able to establish an unconscious mental 
set within their experimental group which enabled it to solve anagrams more efficiently 
than the control group. In his text on the unconscious, Miller speaks of the "efficiency 
of insightless processes." Miller, among others has also conducted experiments in 
subliminal perception under motivated and non -motivated conditions. In both of his 
studies, he found that his subjects were able to identify forms beyond chance expectan- 
cies when the level of illumination was reduced below the differential limen for each 
subject. 



In our laboratory we have been conducting some preliminary developments of 
concept formation tests from which prior -to-insight scores can be derived. Some of 
our leads have been taken from Bouthilet, whose doctoral thesis with Thurstone has 
been concerned with the same problem. One of these tests is called a Word Series 
test in which the subject is given a stimulus word, which is followed by five additional 
words arranged in multiple choice fashion. The subject is directed to "pick the one 
word which seems to go best with the stimulus word. " The correct response is simply 
the word which can be spelled out of the stimulus word. We have tried out this test 
on a small group and have discovered considerable differences among our subjects in 
the number of trials taken to achieve complete Insight. There is also considerable vari- 
ance in the number of correct responses, beyond chance expectancies, which are made 
prior to complete insight. 



Once a reliable measure of learning prior to insight has been achieved, it is pro- 
posed to test Thurstone’s hypothesis that this is related to creative ability. Initially, 
we considered testing this hypothesis by obtaining data on our subjects by means of a 
biographical data blank in which there would be included questions concerning recognition 
through awards, winning of prizes, having stories approved for publication, being recog- 
nized on committees because of one’s ideational ability, etc. However, w~ have discarded 
this sort of a criterion in favor of considering a more relevant experimentally defined 
performance which can be made to yield scores on novelty of solutions, and the number 
of alternate solutions arrived at, in experimental situations. 



I think that Thurstone’s is the onfy real good definition I've seen where an attempt 
is made to differentiate profound intelligence from creativity - - it 's the only one I’ve run 
across - - especially that could lead to some operational testing. 

Bouthilet’s study falls short of reporting on prior -to -insight scores; hers is mainly 
a descriptive and theoretical kind of a study . We have taken some of our leads from her 
thesis, and we developed two tests . 

One test is a learning series where the subject is confronted for several seconds 
with word pairs; he flips the page and he's presented with a multiple choice word series 
test. We also constructed another test that we called the Word Number test where words 
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went with the digits 1, 2, 3, 4. Bouthilet had the Word Series and we supplemented it 
with this test. The Word Series test in our preliminary studies is more demonstrative 
of this pre -focal thinking than the test we developed, because I think some reasoning fell 
into our test. The crucial element - -if you’re going to construct tests like these - - is to 
have one concept that has to be learned, after which there’s complete insight. Don’t 
structure it so that they learn one thing, then another, then another. In our t^j t there 
were four digits and four classifications of objects, so there is some reasoning going on 
in classifying these things. And I think that is corroborated by a correlation with an index 
of intelligence, where the Word Series test has a low and insignificant correlation with 
intelligence but the Complex Concept Formation test has a high correlation with intelli- 
gence. We have revised the second test so there will be just one concept - -the learning 
of one concept and then complete insjght. In the preliminary study, we discovered ter- 
rific individual differences. This is a 20-trial arrangement; so after each trial it’s pos- 
sible to get a score based upon the computation of the 5% level of significance. You can 
compute the chance expectancy level and add to that the standard error of the 5 % level 
and compute the number of responses a person gives you beyond the level and say here 
Is something solid We’ve discovered that some people go along for 20 trials and never 
get complete insight, but they’re giving you responses beyond chance. 

2- -You ask them after each trial if they have it? 

S--We ask them to state after each trial the criterion they are using to pick out 
the one word that goes with a given stimulus word, and you get the most helter -skelter 
kind of protocol that way. 

2” -And your measure is the trial in which they get it? 

S--Yes, that’s when we stop. 

* 

C--And some don't get it at all, but they have gotten it all the time. 

S--Yes, and that may be the person we’re looking for. 

2 - "There 's a single concept throughout here. 

S--Yes, and I might as well tell you what the concept is. The right response is 
merely a word made up out of the stimulus word. It’s as simple as that. It’s a word 
that can be spelled out of letters in the stimulus word: there’s no unscrambling, though. ' 
The stimulus word is a word like response, for example. Now out of response, what 
could you make? ”On” - -yes, that’s a right response, only it’ll be varied and randomized 
in terms of its position in the multiple choice. 

C--Walk did a thesis at Harvard; I don’t know if he ever published it; he did it 
with Dick Solomon, and Solomon would know . 

S--I think it's important that we do this work, because we don't have to insist that 
creativity should necessarily emerge from conscious control. 
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(J- -Is this the only concept you mention? 

* • * i . , * ( 

S- -They’re presented with a learning series for about 20 seconds, in which they're 
given pairs of correct responses to illustrate the principle . Then they turn the page and 
set to work on the test. They're allowed 20 minutes on the test part. The words differ, 
but the principle does not differ from item to item. 

-Hie words in the test differ from the words given to illustrate the principle? 

S--Yes. That was the biggest headache we had going over the Bouthilet material. 
She had a terrific confounding with words recurring, and we waded through several thou- 
sand words . We ended up with nice, discrete learning series which are separate from 
the test series. So you get a learning series, a test series, a learning series, a test 
series through* a 40 page booklet. And you score them on the 20 test series - - it is pos- 
sible t6 get a configuration - - number of correct responses beyond the 5% level, and then 
the point of insight. The same concept runs fight through all 40 pages. 

Q--This matter of picking out the words -- it's purely a matter of discriminating 
the letters that are in the cognate word; it isn't a matter of intellectual insight ofmeaning, 
that is? 



S- -It correlates a little with intelligence. ' •: 

C- -Pattern, however, rather than thought. 

S — We were after a raw feel, an intuitive choice . Pick the word that seems to go 
with the stimulus word, and, believe me, you hurry them along in hopes that they will 
manifest the pre -focal phenomenon, and they do. 

C--Jn a similar manner you could ask them to pick a thought that would be con- 
gruent; or rather to associate and hide a congruent thought there. 

S--No, we haven’t done that. It’s an interesting slant. If you could simplify the 
thought principles so that they wouldn't start a drift off into esoteric areas -- that would 
be OK. If you attempt to replicate a large number of items, you start to roam into rather 
uncommon areas, where you might get into difficulty and go beyond knowledge. However, 
it is a good idea. 

The problem of validating this kind of performance against a criterion has entered 
our minds, of course, and initially we thought that since we are doing these tests on basic 
trainees --who incidentally fall into higher intelligence levels --we'd develop a biographi- 
cal questionnaire, and we have developed one. We’d go back through the lift span of the 
subject and attempt to dig out aspects of his activities that conceivably could relate to cre- 
ativity. And so we structured a biographical index that had to do with his participation in 
fairs and winning prizes, his participation in handling committees in school, his partici- 
pation in courses which represented an attempt to engage in activities of an extracurricular 
nature because school wasn’t challenging him enough, his participation in do-it-yourself 
stuff where he didn’t write away for designs — he designed it himself --that kind of thing. 



But frankly, we have discarded this and we think that a more experimentally defined cri- 
terion should be used. We haven’t got much closure on this. I've been thinking of, per- 
haps, the criterion performance in one like Rees and Israel used - -they established an 
unconscious set. Then they asked who are the people who can profit most fom the un- 
conscious set? If you read their report of several years ago, you find that the experimen- 
tal people did not have any awareness of where they were getting their information from. 
Now perhaps there are differences among individuals in that some could never profit from 
an unconscious set. So maybe that’s the kind of criterion we need. We could correlate 
the pre -focal score with differences on an anagram unconscious set experiment, after 
certain intellectual variables, which may be related to this anagram solution problem, 
are partialled out. Upon reflection, the anagram performance would not provide a ere - 
£tiVity criterion, but it would comprise another measure of pre -focal thinking. 

If we can hold constant any intellectual variables that we think relevant to that 
performance, we could ask ’’What part of performance is this kind of pre -focal thought 
accounting for?" I think it’s important to consider the intellectual variables and partial 
them out or hold them constant by selecting people who are of a certain level on the sort 
of variables that you think are relevant. In this way we would be giving this kind of elu- 
sive phenomenon a chance to rear its head. 



The Prediction of Differences Between Awarded and Non-Awarded Groups 
of Civilian Employees in the Air Force Incentives Program 

Earlier in die introduction I mentioned that the Air Force maintains an incentives 
program at each air base with the epxress intent to encourage the submission of innova- 
tions and ideas which will improve the general efficiency. Those employees who submit 
ideas receive a cash award for their endeavors, provided an awards committee finds that 
a dollar or man-hour savings will accrue to the Air Force. Notwithstanding the sort of 
tough -minded" criterion that has been imposed upon thi acceptability of an idea, consid- 
erable ingenuity seems to be exercised by many of the employees who contribute sugges- 
tions. The ideas are rated for originality, although their adoption is almost completely 
contigent upon the amount of "pay-off" they will have for the Air Force. After the idea 
is adopted at the air base where it originated, its originator may receive additional bene- 
fits if it is applied to other installations belonging to the Air Force . 



Some of the ideas which are submitted do succeed in effecting considerable savings 
or the Air Force. A maintenance technician may invent a method for reaming out the seat 
o a plumbing fixture and thereby alleviate the necessity for purchasing new replacement 
parts. An accountant may develop a procedure for maintaining budgetary records and ef- 
fect a great reduction in man-hours needed for such a task. A stationary engineer may 
study the wiring diagram of a steam plant and suggest an improvement for eliminating the 
danger of explosion, with his idea having greater cogency for the problem than proposals 
made by representatives of the company manufacturing the equipment. This sort of thing 
happens quite frequently, so that manufacturers of equipment for the Air Force can no 

longer ignore ideas, calculated to improve their own product, but which stem originally 
from Air Force employees. s y 



In the Prediction Research branch, we reasoned that the awarded employees would 
comprise a unique group in which to, study creativity, since ea.ch person would have been 
on the record as having made a recognized contribution to Air Force efficiency through an 
original idea. Upon examining the records of our local awards committee, we discovered 
that the awarded group included approximately 5% of the total working force at Lackland. 
Within this group, there were a large number of "repeaters, " and so we felt that we had 
a stable entity. Our research design, as it was formulated finally, called for a controlled 
study, with awarded and non -awarded employees being matched on four variables; namely 
education, intelligence, job competency as determined by supervisory ratings, and time- 
on -the -job. Testing the awarded group offered no problem. However, identifying the 
matching cases necessitated a rather widespread preliminary testing on intelligence and 
the gathering of die other pertinent matching data. In every case,, we made an attempt to 
select a match-mate from the same working environment wherein the awarded individual 
was employed. 

* * ** , 1 

At this time, I can report to you upon the success we encountered in setting up 

our control group. We have succeeded in matching our two groups -- i.e., comparable 
means and standard deviations - -on three of our variables: namely, intelligence,, ed- 
ucation, and job competency. For our own measure of intelligence, we combined both 
forms of the informational scale from the Wechsler -Bellevue, making a total of 35 ques- 
tions . We did not arrange the items for multiple -choice scoring, but left them in their 
original cast. However, we found that scoring the papers could be carried out very 
swiftly. We did not achieve a match for time -on-the-job. However, we found this to be 
a very "slippery" variable to deal with, since our subjects were forced to rely upon their 
memory to recall the number of years they had been employed in any number of related 
jobs. We nciw have sixty-five pairs of individuals, and are engaged at present in seeking 
differences between the awarded and non-awarded groups on several variables which I 
shall discuss below. 

Before proceeding to the independent variables in this study, however, you may be 
interested in the range of talent as we found it in our creative group. With a possible total 
of 35 on intelligence as measured by the combined information scales from the Wechsler- 
Bellevue, our. awarded group runs the entire gamut, with a mean of 14.25 and a standard 
deviation of 5.98. This finding answers our question of whether it was necessary to search 
for creative individuals in the upper range of intelligence alone. As you might expect, 
there is a high correlation between intelligence and education in this group with the latter 
- ranging from little or no education all the way up to as much as 16 years of educational 
preparation. The mean number of years of schooling is 10.5 with a standard deviation of 
2. 6. The other matching variable, job competency as measured by a supervisory rating, 
is of interest.. It caused us to wonder about the impact of the creative individual upon his 
group or supervisor. Is he resented because of his ideas or respected? Is he accused of 
being a tinkerer? Is he more concerned to earn money .through the awards program than 
to be a good working member? As we all know, ratings can be highly confounded by per- 
sonal and social factors, and we were interested in deriving a supervisory impression of 
the awarded group. Accordingly, the supervisors were assembled and given an orientation 
on the study we were conducting among their employees. At that time we had them accom- 
plish the rating form under our direction. They were cautioned not to consider the ratee*s 
paving received an award in completing the rating. Daily job performance was to be the 
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sole consideration. Our ratings run numerically from 3 to 12, the latter being the total 
possible score, with a mean of 7 . 9 and a standard deviation of 2. 1. We do not have any 
data on an unselected group for supervisory ratings, but my guess is that our creative 
group probably does not depart too strongly from the average. 

The tests we are employing in the study have been selected from the Guilford 
creativity battery, and we are just now beginning to score them and search for differences 
between our two groups. At this writing we have already applied a t-test for comparing 
the two groups with respect to Gestalt Transformation, which is the only test in the 
Guilford battery which is multi -choice structured. After correcting our standard error 
between means for the presence of matching variables, we find no differences between 
the groups on this test. In this test the subject must redefine the use of an object to fit 
a task, such as using a watch crystal to stsgrt a fire. 

In addition to the administration of qur matching variables and the creativity tests, 
we are employing a biographical data blank ‘to which we have given the innocuous title of 
Personal Data questionnaire.- I wonder how many of you have given thought to developing 
a specialized type of biographical blank which would attempt to seek out those aspects of 
life experience conceivably related to the enhancement of creative ability. By this I mean 
the writing of items which do not concern themselves with size of high school, city, or 
family, etc. If you have done any thinking about new material fur biographical questions, 
perhaps you will be interested in the sources of our items. 

The first set of items are fairly routine in nature, and we have labeled this sec- 
tion as containin g extra-curricular items . This title seems to be appropriate, for it 
concerns a variety of activities that one would voluntarily take upon himself and in which 
some aspect of creativity might be manifested. \ 

The second block of items we have labeled simply as family life for we have 
gone to various sources concerned with studying family interaction with a view toward 
making them yield material for drafting items . Perhaps some of you may have read Anne 
Roe's studies of eminent scientists which have been published in monograph form. Other 
than finding some gross differences between the backgrounds of physical and social scien- 
tists, we looked in vain for any threads of consistency running through the family life ex- 
periences of her scientists. A projectivist would have a field day with her Rorschach and 
TAT protocols, but we contented ourselves with examining the biographical interviews 
which are included in the monographs . There we found several leads concerning youthful 
home experiences, such as religious attitudes in the home, degree of control exerted by 
parents, extent of argument tolerated in home, and, generally, a description of parent - 
child activities which could be considered as appropriate concomitants to the develop- 
ment of creative ability. Another interesting source was found iq The Authoritarian 
Personality by Adorne, Frenkel - Brunswik, et al . While I am wary of any gross splitting 
of humanity into two groups, it was interesting to hypothesize that the equalitarian home 
would be more conducive to ideational expression. Since a rigidity score can be derived 
from the A-E scale, it was presumed that the moralistic, repressive atmosphere of the 
authoritarian home would countermand creative development. There are probably several 
different syndromes of family dynamics which could harbor creativity, but it seemed worth- 
while to pin down some of the foregoing leads in terms of specific items. We found another 
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source of cues for drafting items in the study by Cattell and Cabot in their determina- 
tion of intrafamilial attitude dimensions through factor analysis. The effective tone 
generated by parent -child relationships cannot be ignored, for there can be no doubt 
that it plays a strong role in determining intellectual development. 

The final block of items in our questionnaire is concerned with teacher -classroom 
dynamics . School occupies a considerable portion of the day for our youthful population, 
and many factors operate upon the young student in the classroom. For the purpose of 
developing items in this area, we went to those sources v/here rating scales are employed 
to assess classroom dynamics. This is with reference to the work of Thelen and his group 
at Chicago, and the pupil -teacher rapport scale developed by Wrightstone . From these 
scales we developed biographical items to cover aspects of classroom life which might 
relate to creative development. 

In order to derive some preliminary statistics on our questionnaire, it was ad- 
ministered to 400 basic trainees at Lackland Air Force Base . Table 3 represents the 
odd-even reliabilities and intercorrelations for the three parts of our questionnaire. 

It may be of interest to point out some of the items which are the best predictors 
of total score on family life. One item is concerned with the reaction of parents upon 
finding that their offspring had "re -defined" the use of some household items as a toy. 
Another is concerned with the level of interest or irritability manifested by parents in 
hobby items or toys found underfoot. Another good item -total score predictor concerns 
the degree to which parents succeeded in dreaming up interesting projects for their 
children during inclement weather. In the teacher-classroom section those items which 
best predict total score seem to describe an individual who looked forward to interacting 
with his teachers, had no fearful attitudes in school, found his teachers willing to ad- 
just to individual, differences in learning ability, and found a receptive attitude in his 
fellow students. 

In our civilian study, we shall continue to look for differences in the creativity 
tests between our awarded and non -awarded groups. At the same time, we have felt 
that it might be desirable to make a closer examination of the very ideas themselves in 
an attempt to set up a continuous criterion. The range of talent is very great in our 
awarded group, whether we consider it from the standpoint of intelligence or originality. 
This range most surely must be expressed in the ideas submitted by this group. Not only 
does the size of awards earned vary greatly, but we have discovered qualitative differences 
in the ideas, too. It is quite possible that originality, as expressed in the official sub- 
mission of ideas, may have its parallel in tests of creativity. It is quite possible that 
the acceptability of an idea on a dollar -saving basis is not the best criterion for us to 
follow at present. For example in our present study we have taken great pains to match 
an awarded employee with one who is not on record for having ever submitted a sugges- 
tion. In between, there lies a rather large population, which has made suggestions, but 
which has never succeeded in having them adopted. What I am trying to point out is that 
an idea may be highly original and yet fail to be adopted because of the economy -minded 
criterion which is imposed upon it. 
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In my own mind there is the feeling that eventually we shall have to return to the 
basic ideas and evaluate them as regards their creative components. This procedure 
will be difficult because many of the ideas are highly technical. It may be possible to 
place some well thought out rating scales in the hands of small groups of experts who 
would be qualified to conduct ratings after an orientation on what constitutes creativity 
in a suggestion. I cannot begin to tell you how interested the Air Force is in maintaining 
an active incentives program, and I know that private industry is inclined very strongly 
in the same way. There is a fertile field for research in this area, and I hope that one 
of the outcomes of this conference will be an integration of our ideas for fresh approaches 
to the study of creativity in programs similar to the one I have just mentioned. 

Q__Were your people with awards willing to cooperate in the study? 

S--For the most part the awarded group was very cooperative - - a real charming 

group. 

Q--Did you try to study their need for additional money? Was this an important 
source of added income? Did they work on it for that reason? Other people had ideas 
and didn't go to the trouble — for it is a lot of trouble to write up something so you can 
get an award - - so some just have an idea and put into effect and xorget about it. 

S--We talked about that, and in some cases the need for money was dominant, 
and in other cases it's just an expression of the individual. 

C--Of course, lots of people need money who are not getting any ideas. 

C--And many people get ideas and don't bother to go through the work to get the 
money for it. 

S--A11 of these comments appear to fit our awarded group. Many of the people 
in this group have a strong desire to achieve, and they are quite willing to go through 
the administrative "hassle" of submitting an idea officially. Incidentally, the whole 
procedure of submitting ideas has been simplified considerably in the Air Force. It is 
also true that many people go along being creative throughout the day, but for some 
reasons, such as sheer modesty or other interests, they never formalize their ideas 
officially unless encouraged to do so. 

The Predictability of Creative Engineering Design 
Through Tests of Originality 

As the program of creativity research became outlined in our branch, we soon 
became aware of other developments in this same field. There were unique populations 
engaged in creative endeavor in various organizations throughout the country, and there 
were talents being trained which could be identified closely with Air Force needs. We 
were fortunate to make contact with Professor John E . Arnold, who is in charge of the 
Creative Engineering laboratory at M.I.T. I feel so enthusiastic about his program that 
I should like to take a moment to explain what he does. Arnold is a graduate engineer who 
has also had some early training in psychology, which he has never forgotten. I should 
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say that he has as complete a library on creativity as I. have seen. He has always main- 
tained an active contact with industry and has been aware of the demand for creative 
thinking in the engineering field. A few years ago, he instituted a course in creative 
engineering at M.I.T. which was a success from the start. Students come to him with 
three years of training in "thou-shalt-nots, " and he has taken on the mission of getting 
them to elasticize those imaginative processes of mind which seem to have lain dormant . 
Therefore, he has his students indulge in a whole variety of ’’out -of- this -world” prob- 
lems, taking them to mythical planets, and asking them to design products for inhabitants 
who are just as mythical. The net result is a spirited laboratory and some very ingeni- 
ous design. ' 

Arnold has maintained an abiding interest in psychological developments in cre- 
ativity, so we found a receptive audience when we corresponded with him for the arrange- 
ment of a joint study. We were interested in a recognizable creativity criterion, and 
Arnold was interested in determining differences among his students in the dimensions 
of creativity measured by various tests taken from the Guilford battery. Last fall, we 
administered selected creativity tests, an intelligence test, and the biogrpahical ques- 
tionnaire I have already described, to twenty -three of his students. This fall we intend 
to pick up more cases. 1 can tell you that we had a lively testing session with his engi- 
neers, and I believe that the high spirits we encountered were due in a large part to the 
release generated by the unstructured formats which characterize the creativity tests. 

Perhaps you may be interested in a comparison of the sample employed by Dr . . 
Guilford in his factor study and the M.I.T. group with respect to scores on the Guilford 
tests of creativity. Table 4 represents the means and standard deviations for each group 
which Dr. Guilford has kindly supplied us. The AAF population is a select group in itself 
hav ing been screened for intellectual and officer qualities as well as psychomotor ability. 
Some of the similarities are quite striking, notwithstanding the limited sample we have 
thus far and the fact that scoring creativity tests qualitatively is not a cut -and -dried 
matter. We have found that the combined informational items from the Wechsler -Bellevue 
give us very little discrimination with M.I.T- students, since their scores range only 
from 21 to 30, in the test which has a possible score of 35. What a difference this is 
from our awarded group at Lackland which practically occupies the total possible range! 

Each student in the course submits three designs during the semester. Arnold 
calls in a small jury of two or three engineers from the Boston area, and the designs 
are evaluated upon their functionality, engineering soundness, and the merits of their 
basic ideas. The scales for rating the ideas are not considered to be too satisfactory, 
and Arnold has a genuine concern for their improvement. There is a real problem in- 
volved in the development of criteria by which to rate any product, whether for a design 
or other ideational efforts . We will encounter a similar problem in the event we shift 
to an assessment of Air Force employee suggestions in the study I reviewed earlier. 

With industry and government manifesting greater interest in the creative process, there 
should be ample opportunity for the psychologist to ally himself with the technical expert, 
with a view to the development of adequate scales or methods for assessing creative per- 
formance in non -laboratory situations. 



Arnold sent us recently the design scores from the jury - - the creative scores 
and engineering soundness scores of 13 students. Just being curious, we ran some 
rank difference correlations. We get a smattering of insignificant correlations, but 
the low level score from the Consequences test correlates with the averaged creativity 
score from his jury. And if you think about it, the students have got to design something 
for a mythical creature, and as they design it, they must keep in mind the impact of the 
product. And that's what you've got to do on the Consequences test; you've got to be con- 
scious of the results of a given incident. And so we were very interested to find that the 
fluency score from Consequences correlates with two design scores on creativity. I 
don’t know whether it will hold up or not; we'll have to do seme mpre testing and see 
whether or not this sustains itself or whether the whole pattern of correlation changes 
when we increase the number of cases - -because we certainly can't generalize too 
strongly from this pilot finding. We've also given the life experience inventory to these 
subjects and we've also given the combined items from the Wechsler -Bellevue - -the 35 
informational items . 



In connection with this study I'd like to mention something in relation to Mc- 
Pherson's proposals and the necessity for developing rating scales for assessing cre- 
ative productivity, . be it an innovation in chemistry or in design. I think his attempt 
is sincere in expanding and generalizing from the patent office criteria for developing 
rating scales, but I think we're going to be missing the boat if we do that solely. I 
diink we should be doing more thinking about scales that get at aspects of the creative 
product which we think fall into the area of creativity, exclusive of the criteria for 
granting a patent. Arnold is very conscious of this problem. We ran a correlation be- 
tween ratings on engineering soundness and creativity. They are rather high, so I'm 
afraid his jury is including engineering soundness too strongly in their creativity rating. 
We’ve got to get them to think about the rateable creative aspects of the design. In a 
design, did the student adapt a functional principle from a radically different area? In 
other words, in automotive design did they take a teardrop principle and produce a car 
Resign? Frank Lloyd Wright comes along and takes a tree and makes a building by hang- 
ing everything on the center post. You get a re-adaptation there. Is there a combination 
of principles in this design? Is the function retained with parts dropped out? I think, we 
should do a lot of thinking on determining ratable components of creative acts in indus- 
trial laboratories, because there is a strong need. They need a psychologist to work 
with the subject matter person in developing these criteria, and I hope that one of the 
outcomes of this conference will be some future research on defining these areas and 
doing some cluster work or factorial work in an attempt to hammer out an area which 
doesn't necessarily ally itself so strongly with engineering soundness. What I'm trying 
to say is that a design can be a "Rube Goldberg" contraption, and yet I think that if it 
represents a creative idea, it should be recognized as such. 

C-My point in bringing up the patent officer 's approach was that it wasn't 
nearly what you usually expect. It was the "gut reaction" of the German patent officer — 
several different methods of measuring novelty, and lots of things that would be sym- 
pathetic with our notions of what creativity really is . 



Evaluation Studies o L Instructional Procedures Designed 
To Increase Creative Thinking 

A study in the evaluation of Osborn's creative thinking techniques is underway at 
present in the Officer Military Schools at Lackland Air Force Base. Dr. Borg and Dr. 
Zaccaria of the Training Analysis and Development unit are its principal sponsors. We 
have assisted them in the inception of a research design, and they are willing to share 
its results with us when available. I should like to tell you a little about Alex Osborn, 
whose theories are being tested in this study, because his ideas have had considerable 
effect upon training courses in industry, as well as in several universities. Osborn, 
as some of you know, is responsible for the origination of the Creative Education Foun- 
dation. Now there is a fundamental procedure which is carried on faithfully in all of 
Osborn’s classes, and this consists of the so-called ,, brainstorm M technique. Under 
’’brainstorm” conditions, the class is called on for an uninhibited burst of ideas for the 
solution of a particular problem or for the improvement of a device. Later, a judiciary 
phase is instituted, in which the ideas are subjected to critical review and subsequent 
adoption. 

The Osborn procedures seem to be working very effectively in business enter- 
prises such as advertising. However, there are signs lately that they are being ex- 
tended to other areas. Headquarters, ROTC has instituted a short course in problem- 
solving in all college ROTC detachments throughout the country, using Osborn’s text on 
Applied Imagination and the manual which accompanies it . The Air Training Command 
became interested in the whole thing several months ago, and requested that, the Training 
Analysis and Development unit conduct an evaluation study. This study, as it has finally 
evolved, combines the creativity test rationales of Guilford and the instructional proce- 
dures of Osborn. There are other studies designed to test Osborn’s hypotheses, but I 
will detail for you at first what is considered to be the major study. 

The actual study, insofar as its field features are concerned, has been accom- 
plished already, and Borg and Zaccaria will be engaged for the next few months in the 
scoring of tests and the statistical analysis of their results. The research design called 
for the arrangement of two experimental and two control groups. Both types of groups 
were given a pre-test on tests selected from the Guilford battery on creativity. Then 
the experimental groups were subjected to 15 hours of instruction in which Osborn's 
procedures were invoked. For the post-test, a whole array of new tests were developed, 
but based upon rationales similar to those existing in the pre-test battery. Dr. Travers 
and I are awaiting the results with considerable interest, not only because we had a hand 
in the "brainstorming” of the post -test battery, but also because it is the only study we 
can recall at present which has be*. :n designed uniquely for testing the merits of Osborn’s 
teaching procedures. 

Otl,er studies which are ancillary to the one I have just reviewed are in progress 
in the Officer Military Schools. One of these studies is concerned with determining the 
effect upon ideational expression of variations in the employment of the ’’brainstorm” 
and the adjudication of ideas. In one group, ’’brainstorm” and adjudication are being 
carried on simultaneously; in another group, the two are relegated to discrete phases, 
with the adjudication session being introduced only after the group has exhausted its 
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repertoire of ideas. Other studies are concerned with the prediction of success in the 
pre -flight school by tests of creativity. • - ' v 



Implications of Personality Variables and Social 
Variables for Creativity Research - " 

We do not have any definite research studies under way at present Which can be 
considered to fail exclusively into the area of either personality or social psychology. 
Aside from the development of a life experience inventory, we have yet to come to grips 
with these facets of the general creativity research problem; However, it is almost a 
matter of necessity that we broaden our scope so as to include non -intellectual factors. 
In our field studies, it has been necessary to mov6 from organizatibn to organization in 
order to administer our tests, and we have noticed considerable differences between 
working units concerning the attitudes taken by supervisors and other members of the 
unit toward those individuals whose ideas are recognized by awards; Where manage- 
ment is sympathetic to the incentives program, the organization is typified by a large 
contribution Of ideas. Where there is antagonism to the incentives program, there 
seems to be a dearth of suggestions, and considerable effort must be expended by per- 
sonnel in charge of administering the suggestions program to stimulate the flow of ideas. 

There is no doubt in my mind that in psychology we shall have to develop a theory 
of creativity which includes variables related to social conditions ih addition to pursuing 
creativity as something which resides solely within the individual . I think we are deal- 
ing with something which is of the "greenhouse" variety, an elusive pimpernel, which 
will not even bother to show its face unless the conditions are right. Now I am not talk- 
ing about the traditional picture of the artist, staffing in a garret and producing great 
paintings . I am concerned with the more typical picture in our western culture, where 
most creative persons are identified with some group, either having been assigned there 
by force of circumstances or having joined it in order to participate in sorpe communal 
productive or research task. At no time in the past has our society been typified by 
so many research agencies and cooperative groups which are deployed over so vast a 
variety of problems in the physical and social sciences. These developments make it 
imperative that we study the social determinants of creativity in groups. 

It is gratifying to see that attempts are being made to formulate theories of cre- 
ativity which do not ignore social implications . I have in mind an unpublished paper 
written by Carl Rogers, which I believe should receive a wider circulation. In his 
paper, Rogers points out that the creative individual is potentially a dangerous person 
because he can be so completely open to one phase of experience and utterly oblivious 
to other important aspects of his social existence. Another social difficulty that the 
creative person faces, which Ross Mooney points out, is his difficulty in explaining his 
ideas to others; for he may attempt to communicate with his fellow scientists in the lan- 
guage of the artist, and thus be incomprehensible . Barron has shown us that the cre- 
ative person may be more impulsive, independent of mind, and less suppressive than 
his fellow men, and these qualities again may result in social difficulties. What, then, 
are the atmospheric conditions under which an individual can actualize his creativity? 



Rogers has listed several conditions under which organizations might produce more 
ideas of a novel and creative nature. He subsumes these conditions under the general 
rubric of psychological safety, which includes (1) accepting the individual as of uncon- 
ditional worth; (2) providing a climate in which external evaluation is absent; (3) under- 
standing empathetically; and (4) psychological freedom. Rogers further details several 
hypotheses related to these conditions, which could be examined in groups experimen- 
tally. Some day I should like to see these hypotheses tested, for there is a great deal 
we can learn from such studies, which relate to leadership training and determining the 
kinds of group atmospheres that enhance creative expression. 

In the area of personality, there are several leads which look promising for 
future study. Many years ago Kierkegaard proposed that, since creativity was a de- 
structive process in one sense, there might be a tendency on the part of creative people 
to suppress their talents for fear of retaliation or censorship. This whole process he 
considered to be anxiety engendering. In his article on creativity and culture, Stein has 
pointed out that the anxiety component can also be generated by the feeling of lack of di- 
rection commonly experienced by the creative person, coupled also with his inability to 
communicate with others. What I am suggesting is that anxiety may be an important 
variable for us to study in research in creative individuals, since our society probably 
is capable of offering more inhibitory blocks to creative expression than it is in provid- 
ing positive atmospheres. 

Since Dr. Barron follows me on the agenda, I do not wish to infringe upon his 
domain. Therefore, I will mention only in passing some of his research which we have 
uncovered in the literature and which appears to provide promising leads for future 
study. Irefef to his studies on the dimensions of complexity -simplicity and independ- 
ence of judgment. He has worked with other correlates of a perceptual nature, and I 
hope he will be kind enough to describe them. Dr. Stein has recently brought attention 
to the fret that the creative person may have a great capacity for tolerating ambiguity, 
thus being able to effect resolution of a problem despite a lack of homeostasis. It may 
be that we should be examining more closely Frenkel -Brunswik * s "tolerance for ambigu- 
ity drive. M Bruner speaks of the physiognomic vs. literal dimension in which there is 
a sensitivity built up about a person or object beyond what is given physically. Persev- 
eration rigidity is truly the "riddle of personality* " as Cattell terms it, and even more 
so when applied to creative effort. Certainly we should expect the creative person to 
persevere upon a problem without at the same time hampering himself by rigid behavior . 
Dr. Guilford has already measured the more positive components of this same charac- 
teristic in his flexibility factors. 

As I close my talk with the foregoing review of .research possibilities in person- 
ality, I am not unmindful of Ross Mooney's contention that creative people can be intro- 
verts or extroverts, impulsive or steady, neurotic or healthy. However, it may occur 
that we shall discover unique personality syndromes which are concomitants of particular 
types of creative expression, in addition to defining the dimensions of creative products . 
As an example of what I mean, the introvert may be more capable of creating that pro- 
duct whjch emerges typically from reclusion. The extrovert, on the other hand, may be 
adept at "creating" new social behaviors or novel responses to life situations. 
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Summary 



A review has been given of the beginnings of an Air Force program on research 
in creativity. The separate studies reported here represent mainly the activities, over 
an eighteen -month period, for a part of the Prediction Research Branch, Personnel Re- 
search Laboratory, AFPTRC. To a large degree, these studies are concerned with the 
application of tests, selected from the Guilford creativity battery to a variety of Air 
Force problems. In addition to the employment of creativity tests, developmental re- 
search with other test media has been carried on in the Prediction Research branch. 
The following studies were discussed: 

(a) Student-instructors were evaluated upon characteristics considered to be 
demonstrative of creative expression. It was found that facto:: composite scores from 
creativity tests yielded significant multiple correlations with these observed character- 
istics . 



(b) A preliminary trial run has been conducted with concept formation tests 
designed to test the hypotheses of pre -focal thinking advanced by Thur stone. Consider- 
able thought has been given to the construction of an experimental criterion situation 
wherein this hypothesis can be validated in a future study. 

(c) The field testing phase of a control study has just been completed at Lackland 
Air Force Base. This study is concerned with seeking differences between awarded and 
non-awarded groups of civilian employees in collection with the Air Force incentives 
program. 

(d) A study was reported in which predictability of creative design ability is 
being sought by creativity tests. This is a joint study, arranged with Prof. John E. 
Arnold of the Creative Engineering laboratory at M.I.T. 

(e) An evaluation study is being conducted in the Officer Military Schools on the 
instructional techniques of Osborn, which he designed to increase creative thinking. 

This study is primarily the responsibility of the Training Analysis and Development 
unit at Lackland Air Force Base. We have assisted in the research design, which in- 
cludes, for initial and final testing, several tests selected from the Guilford Battery 
and newly constructed tests employing similar formats . 

Observations of creative individuals and their working groups made during field 
testing, together with a review of a pertinent literature, have led to a formulation of the 
creativity problem in terms which include personality dimensions and social conditions. 
Certain promising leads in these areas were discussed. 
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Table 1 



Betas and Multiple R’s Including Intelligence 





Assoc. 

Flu. Orig. 


Idea. 

Flu. 


Spont. 

Flex. 


Sens . To 
Redef. Prob. 


Intell . 


Mult. R. 


GIS (phase grade) 


.460 


-.263 


-.150 


-.146 .260 




.538a 


Aud. Reaction 


.294 


-.140 


-.084 


-.110 


.340 


.443 b 


Prob. Sensitivity 


.320 


-.301 


.116 


-.202 


.236 


.485 a 


Originality 




-.138 


.143 


-.112 


.280 


.324 


Spontaneity 


.300 


-.248 


.190 






.372 b 


Native Ideation 


*231 


-.289 




-.111 


.280 


.417 b 



Significant at 1 % level * 
Significant at 5% level 



Table 2 

Betas and Multiple R's Excluding Intelligence 





Assoc. 

Flu. 


Orig. 


Idea. 

Flu. 


Spont. 

Flex. 


Redef. 


Sens. To 
Prob. Intell. 


Mu':. ; 


GIS (phase grade) 




.357 


-.263 




-.146 


.230 


•479a. 


Aud. Reaction 


.290 




-.169 




.081 




.317 


Prob. Sensitivity 


.439 




-.301 




-.123 




.438 a 


Originality 


. 129 




-.233 


.180 




• 


.252 


Spontaneity 


.302 




-.245 


.190 


-.102 




.369 


Native Ideation 


.347 




-.306 








. 368 a 


Significant 


at 1 % 1 


evel 
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Table 3 



Interrelationships of Three Biographical Areas 



1 . 


Extra-curricular *(52) 


.33 


.21 


2. 


Family life 


(tlf 


i*. 

© 


3. 


Family- classroom 




(82)® 



a o-e reliabilities corrected by Spearman -Brown formula. 



Table 4 

Comparison of Means of Certain Creativity Tests 

AAF MIT 







M 


o 


M 


0 


1 . 


Gestalt Transformation 


12.7 


3.7 


13.7 


2.0 


2. 


Common Situations 


35.3 


10.4 


38.6 


11.2 


3. 


Brick Uses (fluency) 


26.6 


9.4 


27.2 


9.6 


4. 


Brick Uses (flexibility) 


9~3 


5.8 


8.9 


2.0 


.5. 


Impossibilities 


12.5 


3.6 


7.0 


2.6 


6. 


Unusual Uses 


22.1 


6.7 


27.6 


5.51 


7. 


Consequences (low -quality) 


14.9 


6.2 


15.1 


6.0 


8. 


Consequences (remote) 


10.3 


4.7 


16. 6 


5.8 


9. 


Plot Titles (low) 


6.6 


5.3 


10.1 


6.0 


10. 


Plot Titles (clever) 


6.1 


3.0 


4.5 


2.7 


11. 


Apparatus 


15.7 


5.3 


20.1 


5.8 


12. 


Quick Responses 


99.8 


18.7 


96.1 


18.9 



AAF Population 

301 Air Cadets 
109 Student Officers 
410 Total 

MIT Population 



23 Students in Creative Engineering Course. 



THE DISPOSITION TOWARDS ORIGINALITY 



Frank Barron 

Institute of Personality Assessment and Research 
University of California at Berkeley 



S- -There has been a marked tendency in psychological research on originality 
to focus attention upon the single original act in itself, rather than upon the total person- 
ality of tne originator. This is understandable, for the birth and development of the 
original idea are usually more immediately interesting and dramatically vivid than the 
birth and history of the man who had the idea. Newton's apple and Archimedes' tub and 
the well of Erastothenes are thus naturally the circumstances with which we associate 
the remarkable insights of these original geniuses . We do not often ask ourselves whether 
these men were for the most part disposed to express or to suppress erotic impulses, 
or whether their emotions were fluent or turgid, or how subject to intense anxiety they 
were, or how much given to violent action. We tend to disembody the creative act and 
the creative process by limiting our inquiry to the creator’s mental content at the mo- 
ment of insight, forgetting that it is a highly organized system of responding which lies 
behind a particular original response which because of its validity becomes a historical 
event. There is good reason for believing, however, that originality is almost habitual 
with persons who produce a really singular insight. The biography of the inventive gen- 
ius commonly records a lifetime of original thinking, though only a few ideas survive 
and are remembered to fame . 

Voluminous productivity is the rule and not the exception among individuals who 
have made some noteworthy contribution. Original responses, it would seem, recur 
regularly with some persons,- while there are other individuals who do not ever depart 
from the stereotyped and the Conventional in their thinking. If,, then, some persons are 
regularly original while others are regularly unoriginal, it must be the case that certain 
patterns of relatively enduring traits either facilitate or impede the production or origi- 
nal acts . Rather than focusing on the immediate conditions which triggered the original 
response, the present study was concerned with the underlying disposition towards origi- 
nality which it may be presumed exists in those persons who are regularly original. The 
research was directed first of all towards identifying individuals who performed consis- 
tently in a relatively more or relatively less original way. When this had been done, the 
more original were compared with the less original in terms of personality organization. 
Independent evidence concerning the personalities of the subjects was obtained through 
the use of standardized paper -and-pencil tests and through employment of the living-in- 
assessment method, with its emphasis upon observation of the subjects through several 
days of informal social interactions, situational tests, group discussions, psychodrama, 
and the like. The observers were, of course, kept in ignorance of the scores earned by 
the subjects on the tests of originality 



It is a basic assumption of this study that acts are original only in relation to 
some specified commonality. The original must be defined relative to the usual, and 
the degree of originality must be specified statistically, in terms of incidence of occurrence. 






Thus the first cirterion of an original response is that it should have a certain stated 
uncommonness in the particular group being studied. A familiar example of this in 
psychological practice is the definition of an original response to the Rorschach ink 
blots, the requirement there being that the response should, \ithe examiner's experience, 
occur no more often than once in cne hundred examinations. In the present study we 
have proposed to deal with a relatively low order of originality, its limits being set by 
the nature of the sampling of subjects. The subjects are one hundred captains in the 
United States Air Force. Originality as described here is originality in relation to the 
usual responses of only one hundred persons . 

(It should be remarked, however, that unusualness of response need not be de- 
fined in reference only to the responses of other persons; one may produce responses 
which are unusual chiefly in relation to one's own usual behavior, in which case the 
response may be said to be original for oneself. A somewhat broader view of originality 
results from such a conception, for a person who is relatively unoriginal in relation to 
people in general may thus be seen to be original at times in relation to himself. ) 

As we shall show later, some of the one hundred captains in this study proved 
to be regularly original in comparison with the remainder, while others are regularly 
unoriginal in relation to the entire group. Apart from their military status, the sample 
may be described as a group of normal, healthy young men of average intelligence, socio- 
economically of the lower middle class in their pre -military background, and similar to 
young men in general in terms of the usualness and the unusualness of their responses 
to the tests of originality employed in this experiment. 

The second criterion which must be met if a response is to be called original is 
that is must be to some extent adaptive to reality/. The intent of this requirement is to 
exclude uncommon responses which are merely random or which proceed from ignorance 
or delusion. An example of the application of this second criterion may be taken from 
the scoring of one of the measures of originality used in the experiment. The measure 
is a count of the number of uncommon -and -correct anagram solutions to the test word 
"generation. " Many subjects did not hesitate to offer solutions which were incorrect, 
and usually unique. In such instances the application of the second criterion of origi- 
nality was straightforward and decisive. Not all of the tests call for such purely cog- 
nitive responses with unambiguous denotative meaning, however. In the case of ink blot 
tests, for example, we come closer to the problems involved in evaluating fantasy or 
works of art, and verification cannot be had by recourse to a dictionary . Instead when 
the examiner himself cannot see the form pointed to by the subject, he must have re- 
course to other psychologists who have given many Rorschachs and who can be consid- 
ered fairly open to suggestions as to what the ink blots might reasonably look like. Con- 
sensual verification is thus sought for such imaginings. Poor forms or uncommon re- 
sponses which did not sufficiently respect the ink blot reality were not credited as origi- 
nal in this study. 

Eight test measures were accepted here as indicative of originality. They are 
described below. The first three of these measures are taken from Guilford and his as- 
sociates in the Project on Aptitudes of High-level Personnel at the University of Southern 
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California. Three tests had significant loadings on the originality factor in those re- 
searches. Of the remaining five measures, two are derived from commonly used pro- 
jective techniques, the Rorschach and the TAT. Another is a commonly used anagram 
test, and the remaining two tests were devised by the writer. 

The first test from the Guilford battery is Unusual Uses. This test calls upon the 
subject to list six uses to which each of several common objects can be put. It was 
scored for infrequency in the sample under study for the uses proposed. Odd -even re- 
liability in this sample is . 77 . In the second test, Consequence B, the subject is asked 
to write down what would happen if certain changes were suddenly to take place. The 
task for him is to list as many consequences or results of these changes as he can. Re- 
sponses are scored according to how oovious the imagined consequences are. The less 
obvious responses received a higher score. Inter -rater agreement was .71. In a third 
test. Plot Titles B, two story plots are presented and the subject is asked to write as 
many titles as he can think of for each plot. The titles are rated on a scale of cleverness 
from 0 to 5. The number of titles rated 2, 3, 4, or 5, constitutes the cleverness score. 
Inter-rater agreement in this study was .43. 

A fourth measure was the Rorschach 0-plus, which was simply a count of the 
original responses given by the subject to the ten Rorschach blots, and judged by two 
scorers working separately to be good rather than poor forms . Standard Rorschach 
administrating procedure was used and inter -rater agreement was .72. Only those re- 
sponses scored as O-plus by both scorers were counted. 

For the fifth score, which was taken from the Thematic Apperception Test, two 
raters working independently rated the TAT protocols of the one hundred subjects for 
Originality, using a 9-point scale with normal curve frequencies. The inter -rater agree- 
ment was . 70. The sixth test was Anagrams, which I have already described. 

A seventh test is the Word Rearrangement (Word Synthesis) test, developed at 
the Institute of Personality Assessment and Research. The subject is given 50 words 
which are selected at random from a list of common nouns, adjectives, and adverbs. 

He is told to make up a story which will enable him to use as many as possible of the 
list of words. The test can be scored for various kinds of performance, including 
economy, originality, and so on, but the subject does not know how it will be scored. 

In this particular study, it was rated for originality on a nine point scale. Inter -rater 
agreement was . 67. You can get a very clear idea of how original the juxtapositions 
of certain words are . For example, one of the words is "coed, " another is "shapely, " 
and another is "chandelier . " Almost everyone sees a shapely coed for some reason or 
other, but if you see a shapely chandelier then it is more or less original. 

C--If originality shows up in a way that expands, as for instance you said that 
the chandelier reminded you of a bowlegged coed, then I should think that might be an 
expansion and militate against a good score in compression. 

S— The subject is not given any set, either to be original or to be economical in 
expression. However, it is quite possible that a tendency in one direction might operate 
to reduce one's score on some other dimension. 
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The eighth test is a set of ten achromatic blots, again constructed locally and de- 
signed originally to measure threshold for the human movement response. The responses 
were weighted according to their frequency of occurrence in the sample under study, and 
the more infrequent responses received a higher weight. The score was the sum of the 
weights assigned to the subject on all ten blots. 

(J--Was it infrequent human movement responses? 

S--No, simply infrequent response of any kind to the ink blots. 

It is worth noting that all eight of these tests are free response tests. The re- 
spondent is not presented with alternatives devised by the test-maker, but must instead 
summon from within himself his own way of solving the problems, seeing the blots, in- 
terpreting the pictures, and putting together the words or letters. There is considerable 
latitude allowed for self expression and for idiosyncratic interpretation. Furthermore, 
diverse media are presented for the respondent to express himself through. The two ink 
blot tests allow for original visualization, or for original perceptual organization of visual 
forms. The TAT and the Word Rearrangement tests permit originality of verbal compo- 
sition to show itself. Consequences and Unusual Uses call for bright ideas in more or less 
discrete form. Plot Titles evokes sloganistic or epigrammatic originality, while Anagrams 
requires a combination of word fluency and ease of perceptual reorganization. 

Now if originality is indeed a dimension and if some persons are regularly original, 
while other persons are regularly unoriginal, we should expect the inter -correlation of 
these measures to be positive and to be statistically significant . We should not, however, 
expect the coefficients to be very high, for it is reasonable that the dimension of originality 
would have its variance apportioned to several media of expression. Even regularly origi- 
nal persons can be expected to be outstandingly original in only one or two ways. The ex- 
tent to which these expectations are confirmed in the present study can be seen from Table 1, 
in which the Pearsonian correlation coefficients of all eight test measures with one another 
are given. 

Q~~Did you compare the O-plus on the achromatic ink blots that you developed with 
the achromatic ink blots of the Rorschach? 



S— No, I didn’t. I just took the total O-plus, ,jut the comparison you suggest might 
easily be done. 

C- -Because all the others have a lot of verbal material appended to the only percep- 
tual material, and the color might be decisive. 

Do you have only rater reliability on ink blot scores? What I mean is, you have 
no internal consistency reliability index on your ink blots, do you? 

S— No, just the inter-rater agreement. 



Table 1 



Interrelations of Eight Originality Measures 



Test Measures 


1 . 


2. 


3. 


4. 


5. 


6. 


7. 


8. 


1. Unusual Uses 


- 


.42 


.37 


.08 


.17 


.29 


.06 


.17 


2. Consequences B 


.42 


- 


.46 


-.02 


.21 


.21 


.16 


.09 


3. Plot Titles B 


.37 


.46 


- 


.17 


.26 


.17 


.16 


.07 


4. Rorschach 0+ 


.08 


-.02 


.17 


- 


.21 


.03 


-.05 


.17 


5. TAT Originality 


.17 


.21 


.26 


.21 


- 


.36 


.41 


.02 


6. Anagrams 


.29 


.21 


.17 


.03 


.36 


- 


.33 


.38 


7. Word Synthesis Originality 


.06 


.16 


.16 


-.05 


.41 


.33 


- 


.09 


8. Inkblot Originality 


.17 


.09 


.07 


.17 


.02 


.38 


.09 


- 



C--Then, the lack of correlation may be due to the fact that the score has no 
internal consistency. Rater reliability alone is not sufficient to insure correlations. 
That’s another variable, isn’t it? There are two different things there, you see, in- 
ternal consistency and rater agreement. What I'm trying to say is that the fact that 
they do not correlate may be due to statistical reasons. 

S--But there may be other reasons. I'm not saying anything beyond this: that 
when you take total scores derived in this fashion they do not correlate with each other. 

(£—' What was the range of your 0-plus scores on the Rorschach? 

S— I’m not absolutely sure, but I think that it ran from 0 to 12. 

Since it is quite possible that originality is a multi -factorial r ,ension in which 
certain factors bear little relationship to other factors, but yet are positively related 
to the underlying dimension as a whole, it would probably be pre’ . ature to exclude the 
ink blot measures from this battery of tests of originality. Considerable doubts must 
be entertained concerning validity, however, and there is another piece of evidence 
which reinforces the doubt. The staff psychologists who conducted the three day living - 
in-assessments were particularly interested in two theoretically central variables which 
they sought to rate on the basis of their observations. One of these variables was origi- 
nality, the other being personal stability. The correlation between this final overall 
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rating on originality given after three days of observation and the eight test measures of 
originality is shown in Table 2. 



Table 2 

Relationship of Eight Test Measures to Rated 
Originality and to Composite Test Originality 





Test Measures 


9. 


10. 



1 . 


Unusual uses 


.30 


.60 


2. 


Consequences B 


.36 


.59 


3. 


Plot Titles B 


.32 


.62 


4. 


Rorschach 0+ 


.,18 


.38 


5. 


TAT Originality 


.45 


.59 


6. 


Anagrams 


.22 


.62 


7. 


Word Synthesis Originality 


.45 


.51 


8. 


Ink blot Originality 


.07 


.46 


9. 


Staff Rating: Originality 


- 


.55 


10. 


Composite Test Originality 


.55 


- 



Also given in Table 2 are the correlations of the eight measures individually 
with the composite variable which is the sum of the standard scores earned by each 
subject in each of the eight tests. Each test measure is correlated with the compos- 
ite of which it, itself, is a part. 

Q--How many observers were there? 

S- -There were ten staff psychologists. 

Q--What kind of inter -rater correlations were there? 

S--The median was something around . 70; I'm pretty sure that it was some- 
where between . 70 and . 75. 



Q--Was it between any two raters? ... 

S--What we actually did was split them into two groups of five and correlated 
the sum of each of the groups - -it was the easiest way to do it. 

C--I will bet with you that it's not more than . 3 for pairs of raters. 

S--It might v/ell be no more than .3 if you dropped down to pairs instead of 
groups of five. 

Q--If you drop out the ink blots, is it possible that you might have raised this 
median? 

S— Well, let me read my statement of this. I thought, of course, of doing that, 
but. . .here is the way I reasoned. 

# 

Again, however, the ink blot measures have relatively little relationship to these 
composite variables. In spite of this situation both ink blot measures were retained in 
the battery for the purpose of identifying original and unoriginal subjects. The reason- 
ing is as follows: On the fact of it, uncommon responses to ink blots are original acts 
under the definition of originality being employed here. Tendencies toward uncommon 
visual perceptions are, of course, not readily recognized in ordinary social situations, 
since they have to be verbalized to be socially visible. Hence, the failure of the ink blot 
test to correlate with the staff rating of originality based on observations of social be- 
havior alone should be discounted. The lack of a verbal component in perceptual origi- 
nality and its conspicuous presence in the other originality tests may also count for the 
relative independence of the ink blot test and the test composite. Finally, if the ink 
blot measures contribute only error variance to the composite, their retention will re- 
sult in failure of some true relationships to appear, but this will be an error on the 
conservative side, and if they do, in fact, contribute true variance not contributed by 
any other test, they may add appreciable validity to the picture of the personality cor- 
relates of originality. They were therefore retained for the purpose of identifying the . 
original subjects. 

Q--Let in the error variance? 

S- -If they do contribute only error variance, it's still an error on the conserva- 
tive side . 



Q - -Were any of the specific tests correlated at all highly with your rating on 
originality? 

S--Yes, in Column #9 of Table 2, the highest correlate of . 45 was with the TAT 
originality. The Word Rearrangement (Word Synthesis) test correlated .45 also; the 
Consequences B was . 36; Plot Titles B v/as . 32; and Unusual Uses was . 30. The first 
three tests in the table - -namely Unusual Uses, Consequences B, and Plot Titles B - - 
of course, are the marker; tests representing the originality factor in creativity. 
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C - -The score for the test of Word Rearrangement had a heavy loading also on 
expressional fluency, and your rating for originality also had some loading on expres - , 
sional fluency. 

S--A dual criterion was now established for calling a subject regularly original. 
He had to be at least one standard deviation above the mean on the test composite, and 
he had to be at least two standard deviations above the mean On at least one of the eight 
measures. With this criterion, fifteen regularly original subjects were identified, and 
more than half of these fifteen were at least two standard deviations above the mean on 
at least two of the eight tests - -they tended to be that far up on two or three of the tests. 
For comparative purposes, the fifteen lowest scores on the final distribution of summed 
standards scores were selected and all of these subjects also met the criterion of being 
at least two standard deviations below the mean on at least one of the eight measures . 
They will be referred to as the regularly unoriginal subjects . There were thus fifteen 
subjects in each group. 

Q - -Could these unoriginal subjects be high on one of the tests? 

S--They could be, but it would be very unlikely. I don't know the exact answer 
on that, but I suspect that it would be very rare that they would even be above the mean. 

Let me now state some hypotheses suggested by previous work. The existence 
of a very general attitude towards experience of a sort which disposes towards complex - 
ity of outlook, independence of judgment, and originality has been suggested by the re- 
sults of some of the earlier studies that I have reported. It is found, for example, that 
individuals who refuse to yield to strong pressure from their peers to concur in a false 
group opinion describe themselves on an adjective check list as original and artistic 
much more frequently than do the subjects who yield to such pressure. In addition, the 
independent or non -yielding subjects show a marked preference for complex and asym- 
metrical lin$ drawings as opposed to simple and symmetrical drawings. This prefer- 
ence for the complex or asymmetrical has previously been shown to be highly correlated 
with the choice of art as a vocation and with rated artistic ability among art students . 
Also in a recent study in New Orleans, ratings of Che originality of paintings in an ex- 
hibition were correlated . 40 with the scores of the artists on the complexity -simplicity 
measure. Furthermore, on a sample of Ph.D. candidates in the sciences, preference 
for the complex and asymmetrical figures proved to be significantly related to a rated 
originality in graduate work. These correlations ranged usually from . 30 to . 45 on dif- 
ferent samples. This same relationship was found among graduating medical school 
seniors who were rated on originality by the medical school faculty. Other evidence in- 
dicated that the opposed preferences for simplicity or for complexity were related to a 
generalized experiential disposition. The preference for complexity is associated with 
a perceptual attitude which seeks to allow into the perceptual system the greatest pos- 
sible richness of experience, even though discord and disorder result, while the prefer- 
ence for simplicity is associated with a perceptual attitude which allows iiito the system 
only as much as can be integrated without great discomfort and disorder, even though 
this means excluding some aspects of reality. 

Now, from all of these considerations certain hypotheses as to the characteris- 
tics of original persons were derived and put to the test in the present study. The 
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hypotheses and the way in which they were tested or partially tested are described below 
in detail. 

Hypothesis I: Original persons prefer complexity and some degree of apparent 
imbalance in phenomena. The test used for this hypothesis was the Barron -Welsh Art 
Scale, where preference for complex asymmetrical figures earns the subjects a high 
score. 



Hypothesis II: Original persons are more complex psychodynamically and have 
greater personal scope. The test here was a psychiatric interviewer rating on "complex- 
ity and scope as a person" on the basis of a tv r o hour psychiatric interview about life 
history, etc. 

Hypothesis III: Original persons are more independent in their judgments. Here 
we had the Crutchfield modification of the Asch experiment which was reported by him 
at APA last year. The "independence of judgment" scale, which is an inventory scale 
that I developed against the criterion of independence or yielding in the Asch experiment, 
was also used. 

Hypothesis IV: Original persons are more self-assertive and dominant. Here 
the tests of the hypothesis were: first, dominance -submission ratings in a psycho dra- 
matic situation which was especially designed to elicit such tendencies. In other words, 
it was a setup where a standard role player attempted to dominate the subject and the 
subject had to either submit or dominate right back and is rated on this . Another test 
of this hypothesis was the Social Dominance scale of the California Psychological Inven- 
tory, which is a pretty well validated scale for the measurement of dominance in real 
life social situations. Another test was a staff rating on dominance after three days of 
observation of the subjects. Another one was the Phallicism scale of the Personal Pref- 
erence Scale. This scale was intended as a measure of the derivatives and residuals in 
the adult personality of propensities which are highly cathected in the phallic stage of 
psychosexual development. High scores indicate an emphasis on personal power and 
desire for recognition. 

Hypothesis V: Original persons reject suppression as a mechanism for the con- 
trol of impulse. This would imply that they forbid themselves fewer thoughts, that they 
dislike to police themselves or others, that they are disposed to entertain impulses and 
ideas which are commonly tabu, and in general, that they express in their persons the 
sort of indiscipline which psychoanalytic theory would ascribe to a libidinal organization 
in which derivatives of the early anal rather than of the late anal state in psychosexual 
development predominate. The tests of this hypothesis were: first, an index of sup- 
pression -- expression on die Minnesota Multiphasic Personality Inventory. We just 
summed the T scores on Lie, Hysteria, and K, and subtracted from that the sum of 
T scores on Psychopathic Deviation and Hypomania, so that in this index the regularly 
original subjects should obtain lower scores -- this is an index of suppression. Second, 
the Policeman Interest scale on the Strong Vocational Blank. This is bound to be a some- 
what derivative measure of the personality tendency toward suppression of outlawed im- 
pulse, but it does at least reflect the similarity of the subject's interest to those who 
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are regularly employed in maintaining law, order, and civil discipline. If you get a 
high score, you are like a policeman in your interests and should be unlike people who 
are original. 

C--Just as long as yoi don’t have these creative people robbing banks before you 
get through. 

2- -You mean they might be better at thinking of ingenious ways to break into a 

safe? 

jS-- Another test is the early anal and the late anal scale from the Personal Prefer- 
ence scale, and, finally, the Impulsivity scale of the California Psychological Inventory 
and the staff rating on impulsivity based on three days of observation. I have this in the 
table, but the thing about this is that practically all of the hypotheses were confirmed 
and most of them beyond the .01 level. 

The highest single relationship was with the Crutchfield modification of the Asch 
independence of judgment experiment, in which the subject is put under pressure to agree 
with a false consensus . 

C- -About reality - -you don’t have any cases where they establish a false con- 
sensus about some social factor attitude. 

S- -Crutchfield uses both the Asch kind of stimulus material and also a lot of 
others, including social opinions. 

2" ~D° you get a composite score? 

S--Well, he scores it as a test with 21 items and so the scores range from 0 to 
21, and that’s what I used. 

2~ "And he doesn’t really make any distinction between physical and social? 

S--Oh, yes, he makes such a distinction. But the score lused is total score. 

If you score the subjects simply on whether they agree or do not agree with group con- 
sensus on the critical trial, it turns out that there is a very wide separation between 
original and unoriginal, with the regularly original people not agreeing with the group 
consensus. The main point of all this is that the only hypotheses which did not get con- 
firmed here were the psychodrama dominance rating. That didn't work; that is, the 
difference was in the predicted direction, but it was not significant. On the hypothesis 
concerning relative strength of early anal and late anal derivatives, it turned out that 
the unoriginal subjects had higher scores on the late anal scale. The original subjects 
had higher scores on early anal . 

2" "Will you please give us a sentence distinction between early anal and late 

anal? 
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Tests of Hypotheses 
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Preference for complexity. 
















Test la. Barron-Welsh Art 
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12.28 


12.67 


10.69 


2.16 


.02 
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2. 


Complexity as a person. 

Test 2a. Psychiatric rating: 
’’Complexity as a person. ’’ . 


6.40 


1.82 


4.00 


1.67 


3.58 


.001 


3. 


Independence of judgment. 
Test 3a. Independence of 


9.60 


1.67 


8.00 


2.94 


1.74 


.05 




Judgment Scale 

Test 3b. Group pressure sit. a 


5.00 


1.87 


8.60 


1.80 


3.93 


.001 


4. 


Self-assertion and Dominance 
Test 4a. Psychodrama: Domin- 


41.13 


11.70 


38.40 


7.78 


0.72 


.23 




ance rating 

Test 4b. CPI: Social Dominance 
Scale 


36.60 


3.74 


28.87 


4.75 


4.74 


.001 




Test 4c. Staff rating: Dominance 


34.40 


7.10 


25.40 


4.06 


4.05 


.001 




Test4d. SCPI: Self-Assertive - 
ness Scale 


15.73 


1.44 


15.07 


2.74 


0.78 


.22 




Test4e. PPS: Phalli cism Scale 
VDC 


13.20 


2.37 


9.13 


4.27 


3.08 


.01 


5. 


Rejection of suppression; tenden 
















cy towards expression of impulse 
Test 5a. MMPI: (L+Hy+K) - 
(Pd+Ma) 


43.47 


26.24 


58.87 


12.30 


1.78 


.045 




Test 5b. SVIB: Policeman Inter- 
est Scale 


44.67 


9.87 


55.00 


10.81 


-2.61 


.01 




Test 5c. PPS: Early Anal Scale 
IVB 


20.33 


4.57 


17.87 


2.90 . 


1.66 


.06 




Late Anal Scale VB 


23.53 


4.59 


26.80 


4.85 


-1.81 


.05 




Test5d. CIP Impulsivity Scale 


23.13 


7.86 


16.60 


6.08 


1.98 


.03 




Test 5e. Staff rating: Impulsivity 32.27 


6.41 


27.80 


5.42 


4.74 


.001 



For the test of this hypothesis, only eighths in each group (eight Originals and eight Un- 
originals) were available . This occurred because half of the subjects in the study were used 
as controls in the Crutchfield experiment, and hence made the judgments without being under 
pressure to conform to group opinion . 
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S — Well, I don’t remember the test items, but the early anal scale measures 
inter esfin such activities as mixing paint or playing with putty, or very direct things 
derived from psychoanalytic theory with an emphasis not on social interaction but on 
preferred activities . The late anal scale includes considerable emphasis on strictly 
defined things, i.e. , being interested in something that’s clearly defined, well struc- 
tured, and so on. I just don’t remember many of the items. I don't think this is an 
exceptionally well -validated test by any means, but it offered the opportunity to test 
this hypothesis, and there is some merit in it if one studies its correlates. 

Q- -Did you purposely leave out the oral material, or did you just not include it? 

S- -I did not include it. Well, my thinking had shifted on it. I take it that your 
question refers to my earlier hypothesis in the complexity- simplicity work. I didn’t 
adopt it as part of this because I have come more and more to think that there is a very 
general kind of disposition which sums up all these things that have to do with the ex- 
pression of impulse. Let me read my discussion of this very briefly, because I do think 
that there is a way of relating all of these hypotheses and the facts. The five major 
hypotheses in the study have been stated in terms I have drawn directly from previous 
observations. There is, however, another way of looking at them, which permits the 
results to be considered in somewhat other terms and in a broader context. 



We have spoken here of the disposition towards originality, with originality being 
so measured as to be equivalent to the capacity for producing adaptive responses which 
are unusual. But unusualness of response may be considered to be a function as well of 
the objective freedom of an organism, defined as the range of possible adaptive responses 
available in all situations. As the response repertoire of any given organism increases, 
the number of statistically infrequent responses, considered relative to the population 
of all organisms, will also increase. Thus the ability to respond in an unusual or origi- 
nal maimer will be greatest when freedom is greatest. Now, freedom is related in a 
very special manner to degree and kind of organization. In general, organization in 
company with complexity generates freedom. The more complex the level of integration, 
the greater is the repertoire of adaptive responses. The tendency towards organization 
may, however, operate in such a fashion as to maintain a maladaptive simplicity. We 
are familiar in the political sphere with totalitarian states which depend upon suppression 
to achieve unity. Such states are psychodynamically similar to the neurotic individual 
who suppresses his own impulses and emotions in order to maintain a semblance of sta- 
bility. There are at hand enough case histories of both such organizations, political and 
private, to make it clear that the sort of unity and balance which depends upon total sup- 
pression of the claims of minority affects and opinions is maladaptive in the long run . 
Suppression is a common way of achieving unity, however, because in the short run it 
often seems to work. Increasing complexity puts a strain upon an organism's ability to 
integrate phenomena. One solution of the difficulty is to inhibit the development of tLc 
greater level of complexity and thus to avoid a temporary disintegration which would 
otherwise have resulted. Originality, then, flourishes where suppression is at a mini- 
mum and where some measure of disintegration is tolerable in the interests of a final 
higher level of integration. 

If we consider the case of a human being who develops strongly the disposition 
towards originality we must posit certain personal characteristics and personal history 
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which facilitated the development of such a disposition. In our hypotheses the term domi- 
nance was used to describe one trait of the regularly original individual, this may be 
translated as a strong need for personal mastery, not merely over other persons but 
over all experience. It initially involves self-centeredness which in its socialized form 
may come to be known as self-realization. One aspect of it is the insistence on self- 
regulation and a rejection of regulation by others. For such a person, the most crucial 
developmental crisis in relation to control of impulse comes at the anal stage of sociali- 
zation. What our hypotheses have suggested is that there is a positive rebellion against 
the prohibition of unregulated anal production and the carrying of the derivates of anal 
and discipline into adult life. The original person in adulthood often likes things messy, 
at least at first. The tendency is toward a final order, but the necessary preliminary 
is as big a mess as possible. Viewed developmentally, the rejection of imposed control 
at the anal stage is later generalized to all external control of impulse, with the tendency 
toward socially unlicensed phallic activity, or phallic exhibitionism in its more derivative 
forms, being simply another expression of the general rejection of regulation of impulse 
by others in favor of regulation of impulse by oneself. The disposition towards origi- 
nality must be seen as a highly organized mode or responding to experience, including 
other persons, society, and oneself. The socially disrated traits which may go along 
with it include rebelliousness, disorder, and exhibitionism, while the socially valued 
traits which accompany it include independence of judgment, freedom of expression, and 
novelty of construction and insight. 

Well, that’s the end of the paper. I have a lot of other correlates of originality, 
found in later studies, which you might be interested in. 

Q- -Don’t you have a situation here that fits in with the calculated risk framework 
that we were talking about yesterday? How much disorder do you tolerate in order to 
have the necessary elements to recombine at a higher order? At one extreme, you can 
get too disordered so that you can never pull them together, or at the other end you can 
have too little disorder. 

S— You want to maximize the expectation of gain while minimizing the risk of 
ruin, but at some time you have to take a big chance on being ruined if you want a big 
gain - -that’s the main idea there. 

C— I think that the big problem is getting these people to sit still long enough to 
teach them the discipline so that they can express their creativity along some avenue. 

If they are so in love with disorder, they rebel against the order which is involved in 
creating. 

Q--I’d like to ask a question along that line. How do you feel that the creative 
scientist differs from your original person? Does he differ or is he just like all the 
others? It seems to me that the creative scientist is a very special type. 

S--Well, our results suggest that he is not especially different. On the art scale, 
for example, the creative scientist quite regularly prefers the complex asymmetrical fig- 
ures rather than the simple symmetrical. He’s like the original person in general. He’s 
different from artists in many ways, of course; his exhibitionism, for example, takes a 
different form probably. 



168 



C - - Also, here you had army captains, where you expect more anal material . I 
think the oral material might relate more to scientists. 

S- -‘Very likely. 

C - -You said something about the control over people as well as the other parts of 
the environment. It seems to me that the findings should indicate that the mathematician 
and scientist are not very interested in controlling people. 

S--I was saying that the original person wasn't interested in the control of much 
of anything. That is, he isn’t interested in the control of people or . . . 

Q- -But how about mastery over things? 

S--Oh, mastery over experience in the sense of comprehension, understanding, 
insight, etc., but not in the sense of manipulating people, i.e., treating people as things, 
which is what one usually thinks of when one thinks of the control of people. 

Q — Do you feel that a person who is creative in certain fields has some of these 
other fields well organized so that he can then be disorganized in this way in his field of 
interest? 

S--I think for one thing it is a very complex problem of personal identity. Origi- 
nal people somehow reserve themselves, or have a sense of reserving themselves, for 
being creative in some things and ignoring others, shoving other things aside so that one 
doesn't have to be creative everywhere. 

Cl — This may be the toleration that was being discussed a few moments ago. 

Q- -Am I right in assuming that these captains are not really representative of 
a strict regular army type? What I’m trying to say is the assessment was being carried 
on at the height of Korean action and a lot of these people were reserve officers who were 
called involuntarily and representative of a lay population really. Now, do you know the 
proportions of the regulars who stayed on active duty since World War II? I have a sus- 
picion it’s a small percentage. 

S — No, about 60% of them were regular military. 

Q- -Really? Because I know there were officers being flown in who were recallees - - 
involuntary. 

S--Yes, some were recallees, but most were not. 

i 

Q - -Do you have one single high power vocabulary test in any battery? How does 
that correlate with your posited tendency to originality? 

S--Well, may I present the results on that? We used the Terman Concept Mastery 
test, which is an analogy and synonym -antonym test, and is thought by the Terman group 



to be a pretty high level test of intelligence in general, or at least of verbal compre- 
hension. Now we correlated the originality composite with all the other assessment 
measures we had, and then partialled out the scores on ..the Concept Mastery test plus 
the Wesman Personnel Classification test. Here are the variables that continued to 
have pretty high correlations (usually between . 35 and . 40) when one partials out the 
intelligence scores. There are a lot of them, but I’ll just read the general headings. ' 
First, general effectiveness of performance. Here you find tests like charades, i.e. , 
overall performance on charades, a rating of total effectiveness in psychodrama, and 
staff ratings on the overall effectiveness of the man as an officer . Then another big 
dimension was energy, fluency of output. The more original subjects also appeared 
more feminine, at least in terms of interest pattern on tests such as the Strong and the 
MMPI. Again, however, I interpret that as probably being related to this matter of the 
awareness of impulse. 

Q--Yes, but don’t you have arousal conditions in the things they were doing? 

I consider that the assessment makes for an arousal condition, and the feminine com- 
ponent gets aroused in this kind of assessment just by sheer virtue of what you call 
for. 



S — Most of these tests were administered in the field under non -assessment 
conditions . 

2 — Well, let’s forget about where they were given. What I’m trying to say is 
that back at their squadrons, if you take ratings on their performance there, you might 
not find feminity to correlate high with ratings, where you do find it with the tests. 

S--Well, I doubt that, judging from our results in other samples as well. I 
suspect that feminity of interest pattern in men is quite generally consistently related 
to originality. 
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A TRANSACTIONAL APPROACH TO CREATIVITY 



Morris I. Stein 
University of Chicago 



S- -Methodologically, creativity has been studied from two major viewpoints -- 
sociological and psychological. The sociologists tend to argue from such data as the 
simultaneity of invention (Ogburn), the value system of the culture (Barnett), patterns 
of cultural growth (Kroeber), etc. , that there are factors in the environment which 
facilitate or obstruct creative developments. These factors have been discussed by 
the writer at greater length in a paper entitled, "The Cultural Context of Creativity. " 

In this frame of reference the individual and his psychological characteristics are de- 
emphasized, for it is assumed that the social forces have broad enough tolerance limits 
to allow for individual differences. In contrast to the sociological viewpoint, the psy- 
chologist looks to forces within the individual, concentrates on such factors as intelli- 
gence, personality, attitudes, etc., and studies their relationship to creativity. An 
implicit assumption in the psychological framework is that the individual is an "allo- 
plastic" organism, that it can alter its environment, and that it can actualize its own 
needs and potentialities. Consequently, psychologists working in this area tend to 
overlook the social milieu in which the individual creates, although they may utilize 
such factors in interpreting their results . 

Both the sociological and psychological approaches have yielded significant in- 
formation for an understanding of the creative process. But few studies, if any, have 
utilized both approaches in the same research, so that our knowledge of the relative 
contributions of both sociological and psychological data in a single situation is quite 
limited. It is our hope that by the time our research is completed we will be able to 
suggest some answers. 

My purpose in this paper is to present some of the basic assumptions underlying 
our research and then to discuss our analysis of the environments in which our subjects 
work. This will be followed by a brief discussion of our psychological findings, and I 
shall Conclude with a survey of them. Since the research is still in process, I am un- 
able to present results on all the issues that I shall raise. Since this audience is com- 
posed almost wholly of psychologists, I have decided to dwell more heavily in this pres- 
entation on the sociological rather than the psychological analysis, in the belief that the 
psychologist’s predictions can be improved if he is more acutely aware of the environ- 
mental parameters against which he is making his predictions (see Methods of Person - 
ality Assessment). 



Basic Assumptions 

Three of the basic assumptions underlying our approach to the problem of cre- 
ativity are as follows: 

1. Creativity is the resultant processes that occur within the individual. In 
general one tends to judge the creativity of others in terms of the "products" that they 
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have produced, or stated differently, in terms of the "distance" between what they have 
produced and the status of the field before they came on the scene. Such an orientation 
causes us to overlook the fact that creativity is a process. It is a process of hypothesis 
formation, hypothesis testing, and the communication of results. Creativity may be 
manifest in any one or all of the aspects of this process. Some people are "creative 
idea men"; o tilers may not be able to generate the ideas but they are quite creative in 
developing the means for testing them; finally, still others are creative in the manner 
in which they present ideas or findings to others. And to be sure, there are individuals 
who are "high" in all aspects of the process. 

2. Creativity is the resultant processes of social transaction. Individuals effect 
and are effected by the environments in which they live. They do not interact with their 
environments without changes occurring in both directions. The early childhood family 
environment transaction predisposes the individual to creativity or sets up intrapsychic 
barriers to creativity. Later, adult environment transactions similarly encourage or 
inhibit creativity. 

3. For purposes of empirical research our definition of creativity is as follows; 
Creativity is that process which results in "a novel work that is accepted as tenable or 
useful or satisfying by a group at some point in time" (see "Creativity and Culture" 
article by the writer). By virtue of this definition we limit ourselves to studying individ- 
uals who are regarded as creative by significant others in their environment. Some of 
you may regard this definition as "too social" but I submit that almost any criterion in 
this area has its roots in the judgments of others. 1 With these assumptions in mind let 
us now turn to the environments in which our subjects work . 

Subjects' Environments and Roles 

Our subjects are industrial research chemists. The companies in which they 
are employed are regarded as sub-cultu.es or sub -systems within the broader culture 
of systems of our society. Each of the sub-cultures has specific goals to accomplish, 
a prescribed status system, a value system, a system of rewards, etc. With regard 
to some of these factors each sub-culture may be similar to others, but in other re- 
spects it may differ and man: " jst its uniqueness and individuality. The constellation of 
that which it shares and that which it holds as unique constitutes its "attraction value" 
for individual scientists seeking employment. Thus, individuals may be attracted to 
certain companies because of their scientific prestige, because of the opportunities 
they make available for creative research, because of their salaries or social security 
policies, because they are located in areas in which the individuals wish to live, or for 
a variety of other factors. And, by the same token, certain companies may possess 
"negative" or "avoidance" value because of their organization and/or policies. 



^In our research we speak of "manifest creativity" and "potential creativity". 
The former refers to that which is regarded by the significant others as cre- 
ative. The latter refers to those individuals who are not now regarded as cre- 
ative but who on our various psychological tests appear very similar to those 
who are regarded as creative. For these individuals we assume th?t there may 
be certain factors in their environments which may account for this difference. 
We hope to test this assumption in the future. 
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In addition to its attraction value, each company may be said to have "selective 
orientation" (Parsons and Shils) in terms of which it selects and recruits its scientists. 
This is manifest when companies seek individuals with specific scientific or social back- 
grounds which they regard as necessary qualifications for employment. Once within the 
employ of a company a man has several roles to fulfill. For the industrial research 
chemist there are four roles - -the scientist, the professional, the employee, and the 
social role. Since these roles will serve as focal points for our research2 I shall dis- 
cuss them and omit a discussion of status system, reward system, etc. 



The Scientist Role 

As "the scientist" the industrial research chemist, like all scientists, is ex- 
pected to "discover, systematize, and communicate knowledge about some order of 
phenomena. " In his role as scientist the individual undertakes activities, not because 
they will be of benefit to any one who may be considered his client, but because they 
will result in more knowledge. "Scientists, in the purest case, do nothave clients " (Hughes). 

In fillfilling his role as scientist, the individual conforms to the ethos of modern 
science which involves the following four institutional imperatives or constraints (Merton, 
pp. 309-16): 

1. Universal ism . The source and claims for truth are to be subjected to "pre- 
established impersonal criteria. ” 

2. Communism. This refers to the fact that "The substantive findings of science 
are a product of social collaboration and are assigned to the community. They constitute 
a common heritage in which the equity of the individual producer is severely limited. " 

3. Disinterestedness . Science demands objectivity and has no place for the 
personal and subjective motivations of the individual . 

4. Organized skepticism. This last institutional imperative involves "The sus- 

pension of judgment until 'the facts are at hand' and the detached scrutiny or beliefs in 
terms of empirical and logical criteria. 1 " 



The Professional Role 

Overlapping with the scientist role is the professional role. As a professional 
the industrial research chemist has been trained in a specific tradition and "only mem- 
bers of the profession are treated as qualified to interpret the tradition authoritatively 
and, if it admits of this, to develop and improve it" (Parsons). This statement holds true 
for both the professional and the scientist. What distinguishes them is that while the 
latter is concerned primarily with increasing knowledge and in communicating with his 

^This area has been discussed at greater length by the writer in a paper entitled 
"On the Role of the Industrial Research Chemist and Its Relationship to the 
Problem of Creativity . " 
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colleagues, the professional earns his livelihood by giving what Hughes has called 
"esoteric service” to a client. The client for the industrial researcher is "the company. " 
By accepting a position with a company a researcher both mplicitly and explicitly accepts 
the task of working on problems related to the products that the company produces. But 
the company is not only the researcher's client: it is also his patron in that it provides 
him with the financial security, the equipment, the personnel, etc., to carry out his 
work. It is the client -patron role, the company vis-a-vis the researcher, that puts 
certain restraints on the fulfillment of the scientist role for the industrial researcher 
and is manifest in the following: 

1 . Limited communism. While the scientist role demands that procedures and 
results should be shared with the scientific fraternity, the professional role demands 
that they are to be shared only with certain selected individuals whose number may vary 
from none to many - -but never outside the company. To be sure, this is a function of 
time, since once a company has secured the patent rights to a process or product it may 
then permit its employees to discuss it. Until such a time, even papers to be presented 
at scientific meetings often have to be "cleared” by the company's patent office, to pro- 
tect the company's interests. 

2. Focused truth. Following the institutional imperatives of the scientist role, 
the individual is free - -as well as obligated -- to pursue the problems and unknowns which 
arise in the course of his work. He need not encumber himself with the artificialities 

of practicalities of the differentiation between pure and applied research. In the indus- 
trial system, however, the goal of each man's work is to be focused on the product or 
products that can be produced and sold by the company to the consumer . Furthermore, 
the best possible product need not be developed at one time, since there are always pos- 
sibilities for "new and improved” products. 

3. Selflessness. The researcher who has many ideas has to be capable of 
yielding them to others. This decision is often arrived at by both the administrator of 
research and the researcher himself, although it is possible that the administrator alone 
may assume such responsibility. At such times it is often necessary that the researcher 
withdraw any self- involvement in his ideas, even though, if they do not work out, his 
reputation may suffer when the people to whom his ideas are assigned make mistakes 
which he might not have made and thus invalidate his ideas. The need for selflessness 
also exists when a project or problem has been completed. At such times the products 
of a man’s labor are sent to the pilot plant and additional alterations may be made. 

4. Communication with lay personnel. The industrial researcher, by virtue of 
his client -patron relationship with non- scientific personnel who are in decision-making 
positions, must be able to communicate with them in non -technical terms. Emphasis 
on this type of communication may occur early in the research process where the re- 
searcher needs to convince management of the value of his ideas, as well as at the end 
of the research when his efforts must be condensed into the ubiquitous "one page or less" 
so that management thinks that its investment has been a wise one. 

5. Vested interest. The professional role demands that the industrial researcher 
be loyal to and maintain the interests of his company, his division, department, section 
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or work group. At scientific meetings or in contacts with customers, the industrial 
researcher is expected to support the vested interest of his company. He must keep 
his eyes and ears open to see how the company can be of greater value to its customers 
and how it can maintain its position among, or surpass its competitors . 



The Employee Role 

The third role for the industrial researcher is the employee role. It is to be 
distinguished from the two previous roles in that their adequate fulfillment adds new 
information to the system, while the performance of the employee role pertains to the 
flow of already existing information and to the maintenance of the system as an ongoing 
enterprise. The industrial researcher shares this role with others in the company. 
Some of the factors involved are: 

Consistent produc tivity. The man on the job must produce with some degree 
of consistency. To be sure, it is expected that the consistency will be a function of the 
difficulty of the problems that the man is investigating. But even on the most difficult 
problems he is expected to show progress in the course of his work. 

2- Financial awareness. From the planning stage through the production stage, 
research and development cost money. The time it costs to sit and think is budgeted 
and charged for like equipment. The researcher must always be aware of the costs of 

his activities as well as be concerned with whether or not his results "will ring the cash 
register. M 

3. Efficiency . Since time, equipment, and personnel are costly, the industrial 
research chemist is expected to be quite efficient in all his undertakings. The best 
idea and working procedure is one which results in a novel product that requires a mini" 
mum of retooling and re- allocation of personnel and functions. These ideas which re* 
quire much shifting about of personnel may generate morale problems, and those ideas 
which require the expenditure of large amounts of money may involve many groups of 
individuals in the decision-making process who must give their approval on the problem. 

4 - Accepting status position and adjusting to authority . While in his research 
efforts the industrial researcher may be iconoclastic and even defiant of authority --i.e. 
exploration of areas in which the authorities in his field say that certain things are im- 
possible --the employee role demands that he accept the limitations and circumscribed 
power assigned to his status position. He has to go through channels and work through 
others in more powerful positions in order to get what he wants. If he is openly defiant 
of those above him, he may well jeopardize his position. He has to learn to adjust to 
them or to get around them without too frequent or open conflicts . 



.Regularity and flexibility. Although the industrial researcher may work in- 
dependently on his own research and set his own rules in this area, he is nevertheless 
part of a working community and he must abide by the rules and regulations that affect 
the total working community. He must attend his job regularly and be there for the 



prescribed working hours. He may not have the opportunity to pursue a "hot idea” 
after "closing time" for to do so may require special permission and clearance from 
the safety engineer and the nightwatchman. The researcher is required to keep accurate 
records of his research efforts because of patent office requirements and also for cost- 
accounting purposes set that the total cost of the development of a product may be cal- 
culated. Within this emphasis on regularity there is the emphasis on flexibility. The 
research man himself may come up against a problem on which he requires the efforts 
of another person, or some one else may come to him for aid, or a problem may arise 
in production that involves a previous problem of his, and for these and other interfer- 
ences the researcher has to be in a position where he stops what he is doing for a reason- 
able amount of time and helps them. 



The Social Role 

The social role refers to the behavior Patterns that an individual is expected to 
manifest in his interpersonal relationships wun superiors, colleagues, and subordinates. 
The individual’s social role varies as a function of his position in the company's status 
hierarchy. The higher the status position the more immunities and privileges accrue 
to him, and he might even be able to alter the role so that it is more congruent with his 
own personality. At lower levels in the status hierarchy, however, the individual may 
feel it is impossible to alter the role to suit his needs . 

The social role differs in one very critical aspect from the roles considered 
previously. For the scientist, professional and employee roles there are usually 
either written or verbalized codes and regulations with which the individual may ac- 
quaint himself. But for the social role the prescriptions are not codified and not ver - 
balized. One learns about them through personal experience, or the individual may be 
informed about them by close friends. At times when they are verbalized in a profes- 
sional discussion, they may be denied, for the social role includes the "irrational" 
factors in the social process with which the scientists does not want to concern himself, 
especially since he may not be too adept at fulfilling them. Yet fulfilling the social role 
adequately is a prerequisite for establishing smoothly functioning communication net- 
works that facilitate one's work and often gain for him the opportunity to be creative - - 
a factor that has often been overlooked in research in this area. 

The discussion of the social role that follows is not intended to be all-inclusive, 
nor does it attempt to account for all the nuances and variations that occur as a function 
of time in the company or status position. Indeed, the frame of reference for this an- 
alysis is the hypothetical individual who "succeeds" -- i. e. , rises in the administrative 
hierarchy, while no single individual may fulfill all the requirements, the more success- 
ful ones are able to fulfill most of them. 

The expectations with regard to social behavior are: 

i. The industrial researcher is to be assertive without being hostile or aggres- 
sive. 



2. He is to be aware of his superiors, colleagues and subordinates as persons, 
but he is not to get too involved with them as persons. 
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3. He may be a lone wolf on the job, but he is not to be isolated, withdrawn, or 

uncommunicative. If he is any of these he had best be creative so that his work speaks 
for itself. 



4. On the job he is expected to be congenial but not sociable. 

5 . Off the job he is expected to be sociable but not intimate . 

6. With superiors he is expected to "know his place" without being timid, ob- 
sequious, submissive, or acquiescent. 

7. But he is also expected to "speak his mind" without being domineering. 

8. As he tries to gain a point, more funds, or more personnel, he can be subtle 
but not cunning. 

9. In all relationships he is expected to be sincere, honest, purposeful and dip- 
lomatic, but not unwilling to accept "shortcuts" or to be inflexible and Machiavellian. 

10. Finally, in the intellectual area he is to be broad without spreading himself 
thin, deep without being pedantic, and "sharp" without being overcritical. 

The scientist, the professional, the employee and the social roles are then the 
roles of the industrial research chemist as we have observed them. There are differ- 
ences both between and within companies in the extent to which any one or a combination 
of these roles is emphasized. Unless we are aware of the character of the roles that 
our subjects are expected to fulfill, significant factors that contribute both the neces- 
sary and sufficient conditions for understanding and predicting creativity are apt to be 
overlooked. For example, if a company is one in which a high value is placed on "basic" 
or "pure" research and a man is expected to fulfill scientific -professional roles, then 
the constellation of factors that would predict creativity in this environment would have 
greater weights placed on a man s theoretical values, capacity to deal in abstractions, 
and independence than would be the case in a company where there is greater emphasis 
on professional -social roles. By considering both companies as a single unit, signifi- 
cant differences "wash out, " and we do not learn the whole story. 

Since our research is still in process, I cannot at this time support my remarks 
on the importance of studying roles with quantified data; the evidence we have to date 
does, however, point in this direction. Furthermore, despite the fact that I have not 
presented specific results on the issues I have raised, I have seen fit to discuss them 
at length for there is some danger in this area of study of applying the knowledge gained 
about creativity in academic situations (e. g. , from the study of students) directly to 
other situations without a prior investigation that would determine whether the require- 
ments of both situations are the same. 



Psychological Factors 



Up to this point in my discussion I have considered only the environmental side 
of the creativity issue. I should now like to turn to consideration of the psychological 



factors related to creativity. To bridge the gap between what I have just said about the 
environment and what is to follow, let me state the issue that confronts us. Assuming 
that an individual is in an environment where the emphasis is on scientific creativity 
and where he has already achieved the opportunity to be creative, what are the necessary 
psychological characteristics that he should possess in order to be manifestly creative? 

To answer this question we concern ourselves both with the man's history 

as well as his present psychological status. Some of our basic assumptio this 

area, are: (1) The creative individual has a personality structure which may be charac- 
terized as an open system with minimal entropy and which is unencumbered by exces- 
sive homeostatic mechanisms . 3 (2) The creative individual has been exposed to greater 
complexity in the mother-child relationship which he resolved successfully early in life 
by detaching himself from others, by relying on himself and by a greater cathexis of 
thought processes. (3) The creative person is a more differentiated individual whose 
behavior is determined by his own value -hierarchy. (4) The creative person has a 
more positive picture of himself md is capable of tolerating “he ambiguity that is in- 
volved in long-term research. 

Now to the empirical work. Our subjects are industrial research chemists . 
They were selected for us by their superiors who ranked them on creativity as defined 
previously. These rankings were corroborated by ratings from their colleagues and 
subordinates. Each man was studied with a variety of psychological tests that were 
either designed or selected to yield information on the variables with which we were 
concerned. Sixteen men were studied in one company and thirty men were studied in 
another. Approximately two days were spent with each man either in individual or 
group testing procedures. Our data are broken down in terms of the results obtained 
from the "more" and "less" creative individuals within each company, and then the 
findings are collapsed so that we can study both company (sub-culture) differences as 
well as "more" and "less" creative differences independent of companies. The results 
that I shall present are those which are independent of differences that are obtained 
both within and between companies. 



Some biographical results 



1. Parents of our more creative subjects have not gone as far educationally as 
have the parents of our lesser creative subjects. 

2. The socio-economic status of the parents of our more creative subjects is 
not rated as high as the socio-economic status of our lesser creative subjects. 



3 All of our assumptions are "relative statements" comparing the "more" with 
the "lesser" creative individual. 

4 

The research is sponsored in terms of a grant-in-aid from the Committee on 
Research Personnel of the Industrial Research Institute. 
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3. Our subjects do not differ from each other in terms of the geographical areas 
in which they were brought up. Both groups are equally likely to come from farms, sub- 
urbs, and cities. 

4. Our more creative subjects say they were more distant from either parent 
and from adults in general than do our lesser creative subjects. 

5. Our more creative subjects say that their parents were more inconsistent in 
their attitudes toward them than do our lesser creative subjects. 

6. While our more creative subjects say that they engaged in solitary activities 
early in life, the lesser creative subjects say that they engaged more in group activities. 

Results related to present psychological status 

1. Our more creative subjects are more autonomous individuals and they see 
themselves as more different from their colleagues than do our lesser creative subjects. 

2. Our more creative subjects have attitudes which suggest that they strive for 
more distant goals than do our lesser creative subjects. 

3. Our more creative subjects have more integrative attitudes than do our lesser 
creative subjects. 

4. Our more creative subjects are more cautious and realistic than our lesser 
creative subjects. 

5. Our more creative subjects are more consistent in their desires for rewards 
than are our lesser creative subjects. 

6. Our more creative subjects have a more differentiated value -hierarchy than 
do our lesser creative subjects. 

7. More of our creative subjects regard themselves as assertive, authoritative, 
and as possessing leadership ability. More of our lesser creative subjects, on the other 
hand, regard themselves as acquiescent and submissive. 

These, then, are a sample of our findings to date. As our analysis of data pro- 
ceeds I hope that I shall be able to present results related to the psychological charac- 
teristics necessary to fulfill the specific roles I spoke about as well as those related to 
creativity in the different branches of chemistry. 

C--Your statement about efficiency in your discussion of the chemist’s role re- 
minds me of one of the stumbling blocks in the Air Force program. A man ccmes up 
with a terrific idea, here at point B, which is highly original. Well, instituting the idea 
results in changes at points A and C . Result: the savings that he would effect here are 
washed out by the costs in retooling there. The idea is then rejected, and the creative 
man is dejected. 



o ' 
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C--In part of your paper you presented a trying picture of the scientist in indus- 
try. Vou must have interacted with industrial research people at Arnold’s creativity 
seminars, and I've seen some of the results of some of the get-togethers of some indus- 
trial managers. Even with the trying picture that faces these people, I have the feeling 
from reading the seminar notes that industry has outstripped the social scientists in 
worrying about this situation. I mean the managers themselves, not the scientists. 

They worry about the role of the scientsits . For example, one of our staff came back 
from a creativity institute, and there one of the industrial managers of a * t search lab 
said: ’’We've discovered, gentlemen, that when one of our men may have l bad idea, 
we have got to let him pursue it. If we don’t, that man is lost to us; that idea will go 
running around in his mind, and so we just have to gamble on him, hoping that he'll have 
more good ideas than bad iders, but he has got to get closure. ” I think that is a very 
profound realization in industry. 

S--Yes, I am inclined to agree with you that industry is becoming more aware 
of the problems their men face in creativity. I know of one industrial research labora- 
tory that had a psychologist on its staff to meet with the researchers in individual coun- 
seling sessions in an effort to help them overcome their personal problems. 

Q- -May I ask what your criterion measure was? 

S- -I should like to expand on that since that is not covered completely in the 
paper. What happened was this. A meeting was held with the Director of Research 
and his immediate subordinates in each of the two companies on which I presented data. 
During the course of this meeting* the whole issue of what creativity meant to them was 
discussed. After the discussion was summarized, each of the superiors was asked to 
rank -order the men who worked for him on a global variable -- creativity - - which they 
would consider as involving hypothesis formation, hypothesis testing, and the commu- 
nication of results. After the separate department rankings were completed, the group 
met together again and rank ordered all of the personnel. This was not as difficult as 
it sounds, since each superior also had had contact with the men or had known about 
them from other laboratories. Thus, a final rank order, which became our criterion, 
was established. In one company 37 men were rank -ordered in this fashion and in an- 
other 102 men had to be rank-ordered. Some four weeks after the initial rank -order 
had been established the same procedure was repeated to establish reliabilities. The 
correlations between first and second rank orders were +. 94 and +. 89 (significant at 
the . 01 level) for each of the companies. To check further on the criterion problem 
we then asked the colleagues and subordinates of the men to rate each of the subjects 
on five variables: number of ideas, quality of ideas, effectiveness of communication, 
creativity and likeableness. In one company colleagues differentiate the more from 
the less creative individuals on two variables - -quality of ideas and creativity. In the 
other company they were significantly differentiated from each other on all four vari- 
ables. On a fifth variable, likeableness, there was a significant difference in the first 
company between the ratings obtained by the more and lesser creative individuals; the 
lesser creative persons were more liked than the more creative ones, while in the 
second company there were no differences . I should also have indicated that we asked 
the men who fell into our criterion groups to rate themselves on the sai^ie variables that 
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theijr colleagues and subordinates rated them on. With one exception the differences 
here are not significant, but they all tend to go in the same direction; the more creative 
men tend to rate themselves higher than do our lesser creative subjects. 

Since we could not study all of the men in each of the two companies, we selected 
the upper and lower 31% in one company and the upper and lower 23% in the other com- 
pany. 

Q - -How do your subjects compare in terms of status in the company as well as 
in the amount of time they have been there and their ages. 

S--Both more and less creative men in both company populations are in their 
early thirties. On the average both groups have been in the employ of their companies 
for 6 years. On neither variable are there significant differences between both groups. 
In terms of status more of the more creative individuals have higher status in their 
companies than is true of the lesser creative individuals. 

Q — How did you measure the socio-economic status of the men? 

S— The men were asked to evaluate this for themselves, and they were provided 
with a scale that was divided into the following categories: upper -upper, lower -upper, 
upper-middle, middle -middle, and lower -middle, upper-lower, and lower -lower. 



REPORT ON CREATIVITY RESEARCH 
AT THE UNIVERSITY OF CHICAGO 

B. S. Bloom 
University of Chicago 



Point of View 

S^-The Examiner's Office of the University of Chicago was created in 1931 to 
prepare comprehensive examinations and other tests required in the New Plan College. 
Over the past 24 years, the Examiner's Office has assumed increasing responsibility 
for the development of a great variety of tests and test research as required by the 
College, Professional Schools or Divisions of the University of Chicago. During the 
past 15 years, we have become increasingly interested in understanding the nature of 
problem solving as it relates to both learning and testing. Through our effort to under- 
stand problem solving we have come to recognize some o ’ the ways in which personality 
influences problem solving as well as the entire learning process . 

We have always taken the point of view that examinations and the examining 
process must be intimately related to the educational program of the institution. As a 
result we have attempted to construct tests and report and interpret test evidence in 
such a way as to have maximal value for both the evaluation of the educational program 
and the appraisal of student progress. Our primary test and research functions as an 
examiner's office at present inclide the selection of students; the appropriate placement 
of students and the appraisal of their achievement at various stages in their educational 
career; the use of the test construction process to help clarify educational objectives; 
research on the relationship between the achievement of students and the learning ex- 
perience they have had; and the use of tests to study various hypotheses about the learn- 
ing process. 

We are interested in the problem of the creative scholar from a number of dif- 
ferent viewpoints, but all have so -ie relation to our position as the Examiner's Office 
in the University of Chicago. First of all, we are interested in determining how to find 
potentially creative individuals and encourage them to secure the advanced training 
necessary for the full realization of their talents. Second, we are interested in deter- 
mining the means by which we can best develop and train creative individuals to maxi- 
mize their eventual productivity and creativity. We recognize the possibility that some 
learning experiences which are an integral part of graduate training may be detrimental 
to the individual’s later productivity and creativity. A third interest, that we have as 
part of an educational institution, is the proper placement of graduates in positions which 
v/ill stimulate and encourage productivity and creativity and which will not form barriers 
against the full realization of the individual’s talents and training. 



Previous Research 

About 12 years ago, our interest in better understanding the nature of problem 
solving ’ d us to make a number of studies of the problem solving processes of students 
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as they actually engaged in the attack on problems. For this research, we selected a 
group of individuals who were outstanding problem solvers and contrasted their prob- 
lem solving processes with that of very poor problem solvers. We found a number of 
major differences in the ways in which these two groups attacked problems. They were 
clearly different in the ways in which they began their attack, their skill in analyzing 
the nature of the problem, their skill in bringing relevant material to bear on the prob- 
lem, and the system and care with which they went through the steps to the final solution. 
In addition to these skills in problem solving, we found the two groups very different in 
their confidence in themselves and their problem solving, and in their motivation and 
desire :o attack problems. These attitudes and emotions interfered with the problem 
solving on one group and facilitated the problem solving of the other group. This re- 
search and our research on selection of students and prediction of academic achieve- 
ment have convinced us that cognitive skills and aptitudes can account for only a portion 
of the variance in measures of problem solving or achievement. Rather reluctantly we 
have come to the view that interests, motivation, and personality characteristics account 
for at least as large a portion of the variance in problem solving and achievement meas - 
ures. 

Studies of Creative Versus Non-Creative Individuals 

/ 

Our initial research on problem solving and some of the studies that followed our 
earlier work (Clifford, Walker) led us to an interest in the problem solving skills and 
other characteristics of very creative individuals. We wished to test some of our ideas 
about problem solving at this more rarified level. For this research, we invited several 
panels of chemists and mathematicians to select what they be ieved to be outstandingly 
creative individuals in their respective fields. We selected another group of chemists 
and mathematicians who were not noted for their creativity although they had about the 
same amount of experience and much the same educational background as the creative 
group. 



Q--Were these graduate students? 

S--No, these were outstanding mathematicians and chemists; the group included 
least one Nobel prize winner, almost all were starred men in science, and they aver- 
aged about 35-40 years of age. There was a high degree of agreement between the two 
separate panels on the top creative men in chemistry and mathematics. The non-creative 
were not really selected from the same area. This was a group that we selected on our 
own as giving little evidence of creativity. 

Q- -What kind of a problem did you use --what sort of complex problem? 

S- -Something like 27 tests. These tests were intended to measure fluency, orig- 
inality, perseverance, etc. 

C- -These are not problems, though. You said "problem solving, " and I assumed 
you had some problem situations in which certain elements were to be combined, a so- 
lution derived, etc. 
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S--No, we did not include problem solving tests in that sense. 

We compared these two groups on a series of tests which were selected to meas- 
ure hypothesized characteristics of creative workers. The results of the comparison 
were that, as far as the tests used were concerned, we could find very little in the way 
of significant differences between the two groups in aptitudes, problem solving abilities, 
or perceptual -cognitive habits. For the 27 tests used in this study, only two yielded 
significant differences between the two groups, and for even these the hypotheses origi- 
nally advanced were reversed by the results. Although we were unable to test our control 
group with personality instruments, several projective tests administered to the creative 
group revealed personality and temperamental characteristics quite similar to those 
fopnd by Ann Roe. At least from a clinical viewpoint, these characteristics help to ac- 
count for the high productivity and creativity of these.- individuals . Thus, we find that 
these individuals seem to have a tremendous amount of energy which they are able to 
channel into productive research efforts . Many of these creative chemists and mathe- 
maticians appear to have some difficulty in establishing warm and friendly relations 
with other people, and there appears to be a need for retreat from the social and per- 
sonal world to a world of ideas and objects. 



Studies in Assessment 

During the past 5 y.ars the Examiner’s Office, in cooperation with a member of 
the Psychology Department and a group of graduate students from the Departments of 
Psychology, Education, and Human Development, has attempted to clarify assessment 
methods of testing and to relate these methods to problems of understanding how per- 
sonality and other affective characteristics of the individual influence his behavior in 
academic situations. We were primarily interested in the way in which the individual 
adapts to the role of a student in relation to the learning experiences provided by the 
institution and the general environment or press represented by the faculty and other 
aspects of the learning situation. We have applied these methods to studies of the ways 
in which physicists, teachers, theologians, and college students learn and adapt to the 
educational environment at the University of Chicago. The basic problem in assessment 
is to find techniques for securing evidence on both the individual and the environment in 
which he operates and to infer from this evidence the ways in which interaction and adap- 
tation take place. In this work we have employed a variety of testing and other evidence- 
gathering techniques, including projective and other clinical techniques, objective tests, 
questionnaires, biographical forms, and interviews. These have been used primarily 
to understand and describe the individual. On the other hand, we have developed a tech- 
nique for describing the significant psychological variables in the environment. These 
have been based upon interviews with faculty, observations of students, and interviews 
with students as they attempt to describe and delineate significant aspects of the environ- 
merit « 



Wo have found these assessment methods to work unusually well as bases for 
predictions of academic achievement or ratings of promise by the faculty. The cor- 
relations between assessor’s ratings and relevant criteria tend to be in the neighborhood 
of +. 80 to +. 90 rather than the +. 60 which we ordinarily get by relating aptitude data to 
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achievement indices. 



Q - -This is an overall rating based on everything he can get his hands on? 

S— Yes, the method of arriving at the rating is reported in a book - -Methods in 
Assessment, by Stern and others. ~ 

Q--What is the criterion? 

S--Xn some cases they were grades and in some cases ratings by faculty. I 
think maybe all I’m trying to say is that assessment methods seem to work both for 
purposes of prediction as well as for understanding the characteristics of the individual, 
the characteristics of the environment, and the nature of the interaction between the two. 
But more than this, these methods yield an understanding of the individual, of the en- 
vironment and of their interactions which can be employed as a basis for selecting stu- 
dents, altering the learning experiences, or otherwise altering the interaction between 
the individual and the learning environment. At another level, this research has en- 
abled us to develop personality types or models which describe groups and classes of 
individuals who are especially effective in relating to the environmental press. These 
methods have enabled us to proceed from highly individual and very costly projective 
techniques and clinical interviews to simpler objective testing techniques or biographical 
techniques which, for certain purposes, yield much the same kinds of insights and pre- 
dictions. In some of our early research we have spent as much as $300 to $500 in study- 
ing each individual. However, as a result of these costly studies we are able to develop 
techniques costing relatively little per subject which may be used to classify and describe 
the individual with little loss in precision. 

It is this last point which I would like to emphasize for the research on the identi- 
fication of creative individuals. While it is quite likely that the initial assessment studies 
of highly creative individuals will be extremely costly per subject involved, there is every 
assurance that these studies can be used as a basis for developing relatively simple and 
practical techniques which can be adapted to large samples of individuals with a maximum 
of validity for the particular purposes of the research or selection. As a footnote on this 
point, we have found it possible to take relatively simple biographical evidence secured 
from questionnaires and by appropriate interpretations to derive a personality description 
of the individual. These descriptions are complete and accurate enough to enable us to 
compare them with the results of extensive personality tests. When all identifying in- 
formation is removed, it is possible to match biographical data with personality test data 
with about 75% accuracy. 



Some Pilot Studies 



During the past 6 months, we have engaged in a number of pilot studies involving 
students and graduates of the University of Chicago. These pilot studies have been done 
with relatively small samples, but selected under such conditions that we are quite con- 
fident that the results and methods are applicable to larger and more complete popula- 
tions of students at the University of Chicago. These pilot studies ape illustrative of 
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some of the types of research that should be done in the future in this field. 



Productivity of Ph.D. graduates 

•* ’ t \ • 

The first set of pilot studies have to do with the productivity of our Ph. D. gradu- 
ates. We followed a group of our Ph. D. graduates who had completed the requirements 
for graduation approximately 8 to 9 years ago . We studied all the bibliographical sources 
to secure a complete listing of the publications these graduates had produced during this 
period of 8 fo 9 years. ' As We summarize this bibliographical data we find that when the 
graduates in t physics and chemistry are eliminated, approximately two -fifths of the Ph. D's 

have produced no research publications other than the dissertation publication. 

. * , . „ * . — 

Q- -How large a group of subjects? 

S— About lbO Ph. D. graduates/' Although there is some variation from field to 
field, with the exception of physics arid chemistry, the variation among the divisions of 
the biological sciences, social sciences, and the humanities is relatively small. Gradu- 
ate institutions produce a very small number of research-oriented people. To emphasize 
this, I would point out that only about 1 out of 3 of our entering graduate students get the 
Ph. D. This suggests that 9 students, enter graduate school to produce one individual 
likely to attain at. least a minimal level of research productivity. 

2« -This is at Chicago - -do you have any corrections for other institutions? 

S--No, I don't. I have reproduced some of these findings in a graph of productiv- 
ity. It will be noted that a few individuals - -less than 10% -- account for approximately 
two-thirds of the research publications of our Ph. D. students, while approximately two- 
thirds produce 4 or fewer publications during the 8 to 9 year period under consideration. 
When we apply a criterion of 5 or mote research publications over an 8 to 9 year pericd, 
we find that approximately 30% of the Ph. D. graduates can be classified as research and 
scholar -oriented while the remaining 70% ihay be said to produce such a small amount 
of research as to be considered non- research -oriented people. 

This type of data, it seems to me, should lead to a re -definition of our problem 
of creativity . One of the possible inferences or conclusions which appears to follow 
from this pilot study is that while productivity is cleanly not'synonomous with creativity, 
it seems quite iikely th^t unless there is some ihinimum or threshold of productivity 
there is little probability or liklihood that the individual is creative. At the upper ex- 
treme of research output, it is possible that high productivity of articles and books may 
be at the expense of creajjivity and. significance. It should be remembered that the re- 
search worker is writing.;and communicating with a particular audience of his colleagues 
and he does have to meet certain standards of research in order to have his work pub- 
lished. Thus, ,1 would like tp suggest that in our research on creativity we consider 
productivity as a ( possible index of creativity. . We must verify this index from time to 
time to insure that we are not going too far astray from our ultimate criterion of crea- 
tivity. But it would seem to me highly desirable that the National Science Foundation . 
research be addressed to 'die problem of identifying and developing individuals who are 
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likely to have a particular level of productivity as scholars and research workers . In 
other words, how can we maximize the proportion of graduates who will become pro- 
ductive research workers? It is my hunch - - and this should be checked by further 
studies - -that the Ph. D. ’s from many other institutions may have even lower levels of 
productivity than that found at the University of Chicago, where research is highly prized 
by both faculty and students. 



We have also checked our original creative and noncreative group of chemists 
and mathematicians and find that in general the individuals who were distinguished as 



creative and non -creative by a group of their colleagues are also very clearly differen- 
tiated in terms of productivity. Members of our highly creative group produced approxi- 
mately 4 units of publication per year over a 10 -year period, while members of our non- 



creative group produced less than a half unit per year. Of course, here we do have two 
extreme groups. Also, there may be many individuals not included in the creative group 
who might have as high a level of productivity as our creative group. 



-When the peers make the rating, they know essentially the activities of the 
scientists? 



S--Yes, there is a great difference between these two groupings - -no overlapping. 



Interviews of faculty sponsors of Ph. D. students 

Another type of pilot study we have engaged in has been to interview a number of 
faculty members at the University of Chicago who are sponsors of Ph. D. students. These 
sponsors have provided us with some anecdotal accounts and analyses of their Ph. D. 
graduates and their eventual careers. What we have tried to do here is attempt to ac- 
count for the productivity and non -productivity of Ph. D. graduates in terms of the ways 
in which the faculty members viewed these students when they first entered the University, 
their ratings and descriptions of these individuals as graduate students, and their analyses 
of particular aspects of their training and eventual employment which helped to stimulate 
and enhance their productivity, as well as the learning conditions and employment situ- 
ations which have tended to interfere with their productivity and creativity. 

Some of the conclusions which ensue from these pilot studies follow: First, 
unusually high achievement in their graduate career is a very good index of later pro- 
ductivity and creativity. However, we do find that mediocre achievement in the gradu- 
ate career may be either followed by later productivity and creativity or lack of such 
productivity and creativity . Thus, we have a situation where we are relatively certain 
of the high achievers in their graduate career but make many errors when we attempt 
to predict from low achievement. 

C--We had some data from a physiology study that undergraduate grades cor- 
relate more with number of papers published rather than with number of citations . The 
higher grades perhaps tended to indicate higher degree of motivation, and emphasis on 
output . 



S--And clearly we are emphasizing here the number of papers published rather 
than number of citations, and there may be some differences. We were using graduate 
grades as predictors of later productivity and creativity. 

Another generalization which seems to follow from our studies of graduate stu- 
dents is that unusually great drive, preoccupation with problems rather than of subject 
matter courses, the attitude of seeking ideas and methods, some independence (although 
not too much or too little), as well as skill in organizing research efforts and analyzing 
problems appear to be related to later productivity and creativity. The lack of some of 
these characteristics seems to be predictive of diminished levels of productivity and 
creativity in later years. Another generalization is that relatively complete acceptance 
of the role of research worker and scholar (rather than the role of student), and efforts 
to find the kinds of training and research experieoc* in keeping with this role appear to 
be most striking predictors of later productivity and creativity. What I'm suggesting 
here is that an individual who comes to a university with problems that he is really in- 
terested in, with some notion of himself as a research worker or scholar, and who is 
able to resist the student role of doing things because they are required or because he 
is told to do them is likely to be a most productive individual in his post-graduate career. 

Q--They tend to take longer getting their Ph. D’s too, don’t they? 

S--No, I don’t think so in this particular population, but it could very well be. 
These are the students who tend to shop around the University and take work because 
they're really vitally interested in it, not just because it's part of the requirements 
toward graduation. 

C— In our physiology data, the "more -citations" people took less time getting 
their Ph. D. ’s. 

As the perception and acceptance of this role becomes ambiguous or competes 
with other roles (i.e., of student, teacher, or the administrator), we find individuals 
who do not produce much in the way of research in their post-Ph. D. career. Another 
factor that seems to be present in differentiating between the productive and non-pro- 
ductive graduates is the lack of competing activities and interests which might inter- 
fere with the role of research worker or scholar. Thus, the family and personal life, 
the social relations of the individual, the quest for status, money, prestige, all appear 
to be relatively subordinate for the re search -oriented individuals, while such competing 
interests and activities seem to be present to a large degree in the graduates who do 
not produce research. Such competing activities and interests may be found both when 
the individual is a graduate student and when he is in his professional career. 

2" “Is it true that the Ph. D. 's here largely enter academic surroundings rather 
than industrial? 

S--A few of them entered government and industry, but most of them are in aca- 
demic situations. Many of these unproductive graduates appear to have great feelings 
of guilt about their lack of research. They usually spend some time talking to their 
sponsor, whenever they see him, telling why they are not doing research this year 
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citing lack of money, heavy teaching load, administrative chores, or the lack of good 
graduate students. Although they appear to have feelings of guilt and inadequacy, they 
are not sufficiently motivated to alter the situation. 

Still another factor which must be taken into consideration is the kind of position 
which the individual finds after his Ph. D. degree. Positions which demand research and 
which supply problems and facilities for research seem to be essential to productivity 
and creativity. Many o i the non-productive graduates appear to be content with positions 
in which the research role is not central, and although they appear to have great feelings 
of .guilt and inadequacy about their lack of research productivity, these individuals do not 
seem to be highly motivated to alter their present duties or to seek and accept, when 
available, positions which would make the research role more central. We do find a 
small number of individuals who seem to be able to resist the demands and pressures 
of their work so as to produce research despite almost adverse circumstances, while 
others working under what appear to be almost ideal conditions for research seem to 
be able to resist such pressures and occupy roles quite different from the research role. 

Finally, this leads us to certain inferences about the system of rewards and re- 
search opportunities available to individuals during their graduate training. Where the 
research role is emphasized, where the individual is given many opportunities to do re- 
search as a graduate student, or where he actively seeks out research opportunities as 
a graduate student he becomes highly productive and creative in his post-Ph. D. career. 
Individuals who seek the student role of ingesting as much knowledge, inform 9tion, and 
skill as possible but whose only research daring their graduate work appears to be the 
Ph. D. dissertation - -and especially a Ph. D. dissertation which is done at great costs 
in terms of time and energy and is not very fruitful - -appear to be pretty well discour- 
aged about the desirability of a research career and regard their Ph. D. dissertation as 
a requirement or hurdle demanded for the degree but which has no real place in their 
lives. Such individuals do not turn out to be very productive in their post-Ph. D. career. 

The general conclusion which we derive from this pilot study is that the research 
role needs to be developed and encouraged during the graduate career and needs to be 
seen as very central in the individual’s employment in his post-Ph. D. career if produc- 
tive scholarship is to result. Where such a role is clear and unambiguous and the indi- 
vidual accepts it, the productivity is high; where the role is confused and where there 
is even great conflict between the research role and other possible roles, the individual 
turns out to be quite unproductive and clearly uncreative in later years . We are of the 
opinion that this type of pilot study should be further developed. The faculty sponsors 
represent an excellent source of information and analysis of their Ph. D. students and 
their eventual careers. If done on a sufficiently broad basis, this can be an excellent 
means for checking various leads to research on the creative scholar. 



Relation between aptitude and achievement indices and graduate school achievement 

Still another type of pilot study in which we have engaged is to study the early 
aptitude and achievement of our college students in relation to their later graduate 
school achievement and promise. We selected a sizable number of college students 



for whom we have aptitude, achievement, and other data; we related these data to their 
graduate school grades and to ratings by the faculty. One of the conclusions we draw 
from this is that general scholastic aptitude does not differentiate high and low achievers 
in the University of Chicago graduate programs. It is quite likely that this is the re- 
sult of the relatively homogeneous character of our undergraduates and the even greater 
homogeneity of our graduate students. All are above a fairly high level of scholastic 
aptitude or intelligence. We do find that specific aptitudes related to the field of spe- 
cialization represent excellent indicators of later achievement. Thus, the physic ii 
science aptitude test is an excellent predictor of achievement in the graduate programs 
in this field. Previous achievement in highly relevant subject matter also appears to 
be a somewhat better index than previous general achievement. By this I mean that 
achievement on the physical sciences or mathematics comprehensive examinations are 
better predictors of achievement in Physical Sciences graduate programs than are the 
overall averages of the students on all the comprehensive examinations. Although high 
aptitude and high previous achievement in relevant fields are good positive indicators 
1 of high achievement in graduate work, students who are lower on either aptitude, pre- 
1 vious achievement, or both may be either high or low graduate school achievers. In 
other words, we can predict from the positive side but cannot find indices which enable 
us to select all of the high achievers and eliminate all the low achievers. When we sub- 
stitute faculty ratings of creativity for grades, we get much the same results. What 
this does suggest is that variables other than cognitive ability or skill are necessary 
for the prediction of achievement in the graduate school . Using these pilot studies as 
well as some of the other research that we have done we would like to make several 
proposals for research on the identification of creative and productive individuals. 

C--That business about the fact that you rarely get false positives, but you often 
j get false negatives --it's clearly in all these cases like the Miller Analogies: it prac- 
j tically never shows up with a false positive. 

| 

S--I think this raises some questions about the selection of fellowship candidates. 
If you're pretty sure that the ones you can select will be good, need you worry about the 
ones you might have missed? 

C— Well, that's where you want to pick up this extra variance. You have to use 
the other predictors at that level and stick with your intellectual measures at the top 
intellectual level. 

S--It may very well be, then, that it does suggest certain screening procedures. 
If you can find suitable measures of cognitive aptitudes, skills, achievements, you may 
not want to go much further with it. But, if other good canditates remain, you may want 
to provide additional screens with other instruments . 

That’s what I didn’t understand. That is, if you took your very creative 
chemists and your non-creative chemists, they were not differentiated on these cog- 
nitive aptitudes, were they? 

S-I was talking about somewhat different tests. The cognitive aptitude tests 
that I was using are general, in some ways like Guilford's tests, whereas che aptitude 
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tests that I was talking about in these later studies are specific to the particular fields, 
such as a specific aptitude test for physical sciences, etc. 

-And you think if you'd had these tests, you would have found a difference be- 
tween the top and bottom creative chemists? 

S--Yes, this is my hunch at the moment. 

Q- -T know you expect this as far as students are concerned, but the big problem 
hers is really whether the student difference actually obtains in life. 

S--X am also trying to find out if some of these data correlate with promise after a 
Ph. D. with about a 10 year follow-up study using a criterion of ratings on creativity. 

-Were these people selected on the basis of the test data? Were the data 
available to the committee that selected them? 

S--Yes. These were all students in the college at the University of Chicago who 
went on to graduate school. I am suggesting that those who were highest on certain spe- 
cific tests in the college were the highest in the gra aate school and at later times . This 
was one basis for selection. 

C- -There were a lot of people cut off that might have been good. 

<C - -But you won't select any duds by using your subject matter tests. 

S — But you will miss some very productive and some very creative individuals . 

0--What are the bets that the next 10 years on the group of whom you painted a 
more or less depressing picture, will show a terrific difference? Aren't they gcing 
through a period where (a) they're loaded down with a lot of undergraduate courses, 

(b) they haven't made a name? 

S--My hunch is that there will be little change. You realize that I did report on 
individuals 10 years after the Ph. D. in the age range 25-40. The picture 10 years be- 
yond that would probably be very similar, according to my best guess. 

Three Proposals for Research 
A descriptive -rating questionnaire 

A very practical attack on the problem of identifying creative students is likely 
to result from the development of descriptive-rating questionnaires. Situational tests 
in which the examinee attacks real problems in the field are undoubtedly excellent tech- 
niques for observing and predicting creativity. However, the difficulty of constructing 
and administering >mch situational tests provides serious limitations. We would assume 
that secondary and college teachers have many opportunities to observe their students 
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in situations which approximate research positions and that they can, if provided with 
proper techniques, report the behavior they have observed. 

The limitations on ratings by faculty members stem from the lack of norms or 
comparison groups against which to rate the student, from the general ’’halo" effect 
when the faculty judge rather than describe the student and his activities, and from the 
lack of sufficiently detailed specifications for the description of the student. One pos- 
sible solution is ,to provide the faculty member with a detailed descriptive questionnaire 
which can be filled in about 15 to 20 minutes for each student under consideration. 
Judgments about the meaning of these descriptions should be imbedded in the scoring 
scheme which is applied after the descriptions are submitted to a central office . 

Some of the steps in the development of such a descriptive rating questionnaire 

are: 

1 . Use research already reported to find descriptive models of highly productive 
and creative scientists :e.g. , Ann Roe, Chicago studies, Ter man's reports, etc. 

2 . Translate these models into behaviors and activities which can be observed 
at secondary or college levels . 

3. Construct a questionnaire containing statements of those behaviors which 
can be used by a teacher to describe individual students. 

4. Test instrument by selecting individuals with high and low promise in gradu- 
ate school and determining whether their undergraduate and secondary school teachers 
in relevant subjects describe the two groups differently. Repeat the procedure with 
graduate school instructors. 

5. Develop scoring methods and standards which maximize differences between 
low and high candidates . 

6. Test technique on more varied samples, including samples of N. S. F. can- 
didates after they have been screened for fellowships . 



Problem-solving tests 



The pilot studies and other research suggest that tests which require the prob- 
lem solving techniques of the scientific field are likely to be better predictors of later 
achievement than tests of subject matter knowledge or tests of more generalized prob- 
lem solving techniques. The examiners at the U. of C. have, over the years, experi- 
mented with a variety of problem solving techniques and have developed a number of 
promising methods which should be applied to the identification of creative talent. 

Problems : It is necessary to provide problems which are complex enough to 
require many aspects of problem solving and which can be described in enough detail 
to start candidates at the same level. The problems must be realistic enough to secure 
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high motivation and a T ^opriate effort from the examinees . Methods must be developed 
which will enable us nake judgments not only about the accuracy and speed of the 
problem solving but aiSo about the adequacy of the methods of attack, and the steps in 
the solution. 



Method 

'* ; • . ■ \ / 

1. Construct several problems which are highly relevant for students completing 
undergraduate work within a particular range of major fields. 

2. Select promising and unpromising candidates on other criteria and observe 
their problem solving processes as they attack these problems. 

3. Develop simplified techniques for administering the tests and scoring the 

results. f 

4. Try these techniques with NSF candidates as well as with other selected 
populations where other criteria are available or can be secured. 

We havqt already tried some of these methods on an experimental basis and feel 
confident that we can develop tests which will enable us to predict the individual’s skill 
in analyzing problems in his special field and in carrying out the necessary steps in the 
research procedure. One technique we have found promising is the "open-book” tech- 
nique, in which the examinee is given reading materials describing the problem and 
providing the necessary background, followed by a series of questions and problems. 

The other technique which appears especially promising is the "tab-sort” test, in 
which the examinee is given a problem and a series of cards. As he selects each card 
in relation to a particular question or hypothesis he may have, he secures relevant in- 
formation and then proceeds to the selection of other cards which form steps toward 
the solution. The task is to solve the problem with a minimum number of cards and 
to select each card as a logical step toward the solution. This "tab-sort" test is in 
many ways the testing equivalent of the popular game of twenty questions, where the 
individual asks questions and receives answers as he tries to reach a solution with a 
minimum number of questions and with a minimum amount of help or answers from 
the opponent. We believe this method, when applied to relevant problems in a particular 
subject field, may yield a very good index of the problem solving processes, habits 
and skills of the individual . 

Assessment studies of graduate students and research workers 

We believe it is highly unlikely that creativity may be regarded as a character- 
istic solely of the individual which he possesses under all circumstances. Creativity 
is, we assume, an interaction between an individual and an environment.. Variables in 
both the individual and the environment may facilitate or hamper creative effort. We 
should like to continue research along che lines already begun in our own graduate 
schools and with creative chemists and mathematicians. 

Using faculty ratings of promise for graduate students and levels of productivity 
for post-Fh. D. research workers, we would like to select outstanding individuals and 



compare them with less promising or less productive individuals. We plan to use a 
battery of clinical techniques, objective tests, and biographical and interview techniques 
to get at personality characteristics. We would also plan to use objective tests of prob- 
lem solving, achievement and aptitude, and perceptual -cognitive habits. 

On the basis of this broad range of procedures we would attempt to determine 
significant factors in the individual’s needs and abilities. Using other techniques, in- 
cluding interviews with faculty and research supervisors, we would attempt to isolate 
significant factors in the graduate school and work environment. Out of this research 
we would attempt to determine the ways in which the individual comes to accept the re- 
search role and the types of reinforcement of this role which takes place during gradu- 
ate training and in the employment situation . 

The adequacy of the models derived from this research would be tested on new 
populations with their effectiveness determined by predictions of selected criteria. 
These models would then be translated into simpler, more economical, and more 
practical techniques and their effectiveness determined on new populations . 

O-It appears that both our tests and our conferences may be too speeded to 
deal fully with creativity. 
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THE CREATIVE PROCESS AND ITS RELATION 
TO THE IDENTIFICATION OF CREATIVE TALENT 

Brewster Ghi selin 
University of Utah 



S-'It has been suggested that I give a brief outline of the c/eative process as it 
seems to me the most dependable information represents it and the most defensible opin- 
ion has defined it, in contrast to popular misconceptions. Roughly this is the view of it 
represented by the Introduction to my anthology The Creative Process , which though pub- 
lished only in 1952 was written in 1946. I will try to define also some ideas transcending 
the theory developed there. 

What may be called the classical picture of the creative process is Henri Poincare 's, 
formulated about fifty years ago in a report of his own processes, in his essay "Mathe- 
matical Creation. " Poincare insists, you remember, on the necessity for preliminary 
labor. At the beginning of the experience which demonstrated for him the ways of his 
mind in creation, he worked for fifteen days on a problem without producing any results; 
then after an interval he was surprised by the spontaneous emergence of the solution, 
which he went on to verify, laboriously but without difficulty. This sequence of labor, 
quiescence, illumination, and further labor has turned out to be common in the experi- 
ence of creative workers of all sorts . 

The first stage of the process described by Poincare^, the preliminary labor, 
must be understood to be something other than that preparatory effort which educates 
the inventor in the known techniques and materials of the field in which the creative 
work will be done. That preparatory labor, so obviously valuable, causes no creative 
activity whatever . To be skilled and erudite is not to be creative, even potentially. 

The labor that is really indispensable appears to be, just as obviously, worthless . For 
Poincare' the creatively significant labor always involved a seemingly fruitless struggle 
for insight into some area of obscurity in the subject of his interest, mathematics . Its 
value lay in its consequences. For always, after a period of quiescence that followed 
this apparently useless work, the obscurity was suddenly dispersed, the difficulty was 
removed, by the spontaneous appearance of fresh insight. 

This advance in understanding, which may be considered the crucial action of 
the mind in creation, was for Poincare', as it is for all of us, the central mystery with 
which explanation must grapple. Poincare' regarded the preliminary labor as an incite- 
ment of the unconscious mind to further work. He supposed that he had found confirma- 
tion for this belief when, lying wakeful one night after some ineffectual labors had been 
followed by his drinking too much black coffee, he observed the apparently spontaneous 
evolution of a new theorem in his mind. Perhaps he was justified in accepting the sub- 
jective actions he observed then as typical of his own ordinarily unconscious processes. 
All we can know, however, is that the crucial actions were rapid, energetic, spontaneous, 
sure -- and conscious. Other theorists than Poincare have conceived of the process, in 
the mysterious second stage, as an incubation or gestation, because it is terminated by 
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the emergence of new insight. But these suggestive terms are Very imprecise, about 
as much so as the notion of an "unconscious” activity of the mind. In Henry James’ 
famous phrase the "deep well of unconscious cerebration" the momentous metaphor of 
depth psychology is easier to recognize for what it is, a picturesque substitute for an 
avowal of ignorance. 

The third stage of the creative process defined by Poincare, inspiration or illu- 
mination, is the sudden, spontaneous appearance of new insight, accompanied by feel- 
ings of certainty, which are not always valid, and of esthetic gratification. The reve- 
lation that occurs has sometimes been compared to a flash of lightning in darkness sud- 
denly exposing an unsuspected structure, the product, Poincar/ would have us believe, 
of prior unconscious work. But it might be more correct to think of the lightning and the 
structure revealed as two aspects of a single event. Possibly the structure makes its 
own light as it comes into being in the mind. What is indubitably sound in Poincare’s 
view, verifiable in terms of the experience of every kind of worker, is the idea of in- 
spiration as an emergence of new insight attended by more or less intense feelings of 
conviction and of esthetic delight. 

The fourth and final stage of the creative process as Poincare describes it is 
verification, for the mathematician a term of obvious import . For the artist this stage 
would involve reappraisals, repainting, revision, rewriting. For the scientist it would 
perhaps require the setting up of experimental situations in which the insight central to 
a hypothesis could be brought under the pressure of actual circumstances, the conditions 
of reality. 

In its mere outline, Poincare's conception of the creative process corrects the 
extremes of popular misapprehension represented in the contradictory ideas that the 
process is all inspiration or all labor. The common notions tend to persist, however, 
in a modified form. To sober ignorance, pondering one and another glimpse of the facts, 
the creative discipline may appear to alternate between madness and method, a mindless 
wildness that looks ludicrously irrelevant to the ends produced and a knowing labor so 
painstaking that it might be endugh by itself to produce anything, no matter how wonder- 
ful. Though both of these concepts have some slight color of truth, neither is accurate 
enough to represent the reality, and together they are useless for explanation or guid- 
ance. For essentially what they purport is that irresponsible divagation combined with 
dogged determination will somehow or other produce creative results. The imperfec- 
tions of this notion will become more apparent as we examine the creative process in 
greater detail. 

Some nourishment for the popular belief that inspiration is a species of madness 
may be found in many authoritative discussions of the creative process. Plato, in his 
dialogue Ion , plausibly represents the rhapsodist Ion as succeeding best when he was 
behaving not unlike a madman. Katherine Mansfield believed that the writer in his best 
moments is "possessed. ’’ Expressions of contrary views are rare enough to be remark- 
able. Chekhov’s objection to all such opinions supports the idea that inspiration is a 
madness. He argued that if anyone claimed to have written a story by inspiration, "with- 
out premeditation, " he could rightly be called a lunatic . Exactly this procedure was 
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claimed for himself by William Bl<?ke, who has often been regarded as mad, though I 
believe unjustly. One of his poems, he reported, was written in section* of twenty 
lines or so at a time involuntarily and ’’without premeditation. ” We may compare with 
this involuntary production Poincare’s sudden seizure as in the midst of a conversation 
he set his foot on the step of a public conveyance; unexpectedly his mind was illuminated, 
flooded with understanding, and then he went on with the conversation. He might have 
been in danger of appearing mad, if he had not deferred his verification of the thought 
that in that instant intruded upon and momentarily possessed his mind. He simply left 

it - - brushed it aside for the time, and returned to the verification at his leisure . 

% 

Another closely related popular idea, that the inventive mind is eccentric, might 
seem to be not wholly a false one. Musicians, like prophets, sometimes wear their 
hair long and unkempt. The image of Einstein occurs to us, moreover. Why do some 
creative people allow themselves to appear eccentric? Perhaps Li order to avoid the 
stereotypy of thought that is suggested by the role of being an average person, perhaps 
to defend a sense of self as potentially other than it seems to be in ordinary moments. 
Yet the eccentric appearance and behavior are themselves a stereotype, purporting 
the popular ideal of creativity. The extremes of this sort of divergence are represented 
in the public exploits of Salvador Dali, who is said to have strolled a lobster on a leash 
along Fifth Avenue. But his act was not original, for somebody else walked a lobster 
in the past century. All such tricks easily become as stale as they are fruitless. Many 
people have thought that by eccentric behavior or even by any exceptional behavior like 
that of creative people — going to the South Seas as Gauguin did - -they could increase 
their chances of becoming creative . Of course creativity depends upon no such things . 

The great prominence of painstaking labors in the creative process occasions 
other misapprehensions. A common delusion among those ignorant of the creative 
process is that it begins in clarity and order, systematic understanding, and proceeds 
in logical advances and under pressure of will to the development of a foreseen, or at 
least partially foreseen, structure or system. Actually it begins in obscurity and in 
some degree of confusion, a state of mind aptly described by Alfred North Whitehead 
as an ’’imaginative muddled suspense. ” The testimony is almost universally to this 
effect, fhe only counter notions among the creative :ome from those who find that 
the process of creation has no beginning at all except a sudden insight, which is usually 
full of ambiguous promise rather than of final revelation. Faulkner in describing the . 
origin of The Sound and the Fury said that the first insight came to him as an image of 
a little girl playing in a stream and getting her panties wet. In its slightness and par- 
ticularity, that beginning is typical enough . Henry James has described the germinal 
trivia that surprised his mind at times as ‘’rich bare little factfs]” in which he would 
feel a strong suggestion of potentiality. Paul Valery reports much the same experience 
of indeterminate promise sensed in some slight element of fresh form. We sometimes 
discover first, he remarks, not the diamond but the blue ground; we feel that we have 
approached an insight; we sense the presence of something which has yet to be appre- 
hended. 



The first stirrings and advances of even che most strictly intellectual creative 
achievements appear to be realized primarily in sensory or passional terms, or in both. 
Einstein has described the early stages of his thought as a somewhat vague ”associativ( 
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play” with incommunicable entities, visual and muscular, which he manipulated in a 
desire for clarification not immediately conceptual but only ultimately so. Poincare 
in his essay ’’Mathematical Creation” argues that the movements of the mathematician’s 
mind in creation are guided by esthetic considerations; and other mathematicians have 
agreed. What he refers to is not feeling in the sense of emotional excitement, bfUt an 
affective response to an intellectual order still eluding rational grasp. In the creative 
process of the arts, incipient configurations are known in exactly the same way, as 
allurements affecting our sensibility. 

Following such intimations as the work continues, the creative mind is guided 
by no very precise foresights . Amid a multitide of shifting images and ideas it must 
settle its course by immediate action. The usual procedure is indicated in Picasso's 
assertion that ’’The picture is not thought out and determined beforehand, rather while 
it is being made it follows the mobility of thought, ” Poincare’s report of his discovery 
of the Fuchsian functions while he lay wakeful after coffee gives the same impression 
of unpremeditated development. Insight was attained by immediate action amid the 
multitudinous motions of thought: ’’Ideas rose in crowds; I felt them collide until pairs 
interlocked, so to speak, making a stable combination. ” 

No sort of calculation from known grounds will suffice for creative production. 
The requisite act is a fresh formulation, not a mere copying with variations, an elabo- 
ration, contraction, or switch. Turning out a new model on an old plan is not really 
creative. It is only an operation. The idea of remaking a mermaid, for example, to 
be fish above and woman below, has occurred to many people. It involves merely a 
reversal of the familiar; it is like so much of the wit of Oscar Wilde, an inversion of 
platitude, with the advantage of disturbing our preconceptions and introducing variations 
of detail . 

No prior determination of the mind’s course is possible, for the order the mind 
is seeking to realize is the only one adequate to direct it. Virginia Woolf has described 
in the introduction to her most famous novel how sue wrote Mrs. Dalloway , not from a 
preconceived plan but on a plan that emerged only in the struggle with the morning's 
work, ”in the act of writing, ” a necessary procedure because the traditional form of 
the novel presented to her a preconception that would not allow emergence of configu- 
rations embodying the insight* toward which she was striving. The plan of the book 
was grasped as an unforeseen configuration realized directly in materials brought 
alive to the mind. 

«» 

In its attainment of fresh configurations, the mind does not move in security 
from the old configurations of its insight to the new ones, extending the grounds of 
certainty as in a march into new territory adjacent to the old and accessible from it. 

It is rather as if the mind advancing where no grounds exist creates new ground under 
its feet. 

A creative product is distinguished from all others by the presence in it of a 
configuration which is intrinsically new. In order to bring into being such a structure, 
not to be found anywhere among the personal or communal possessions of the mind, 
one must turn from all of them. In the exact phrase of Souriau quoted by Hadamard 
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in his book The Psychology of Invention in the Mathematical Fie ld, M In order to invent it 
is necessary to think aside, " to purify the attention cf everything that ordinarily would 
occupy and define it. That idea of diversion seems to relate to the nonfocal conscious- 
ness, to use a term attributed to Thurstone in our talk about it this morning. I have 
another piece of jargon for it: I will call if the preconfigurative consciousness rather 
than the nonfocal --or the preconfigurative state of the consciousness. For what is 
absent, or much reduced, in the state of mind we are concerned with is not focus, not 
distinctness in the form or forms presented, but form itself, in the sense of structure 
that is fixed, determined, more or less resistant to flow, fusion, and dispersal. Henry 
James' "rich bare little fact, " a definite situation involving two people in dispute over 
ancestral furniture, was perfectly in focus, such as it was, from the moment it came 
to him. The forms of fresh insight do not emerge gradually out of a vague but determi- 
nant background, as when the faint streaks and blobs of a blur staining a screen darken 
into clear lines and areas. They occur clear cut in the rich welter of the mind roused 
by its avidity for an assurance not defined in any of the existing expressions of its cer- 
tainty, in any of the multitude of known configurations, all of which it tends to dismiss 
out of indifference to their inferior relish. The old, irrelevant configurations are re- 
linquished, nU rejected in a voluntary effort that by concentrating attention on them 
must restrict the movement of the mind. And thus they fade from attention spontane- 
ously. 



C — Brewster, I'm reminded that Thurstone brings out the point that we can't 
will what he calls dispersed attention. Solutions of problems come to him at the most 
odd moments. And you get the feeling as you read some of his material that he wishes 
there were some way that he could enforce the dispersed attention upon himself. 

iC--Take mescal. 

iC — It's not just the physical, it's the mental tension that gets involved here. 

S--I believe, Frank, that you said you were going to experiment with physiologi- 
cal conditions. Are you going to experiment with the creativity of drugged subjects? 

C--We hadn’t thought of doing so; it's a good idea, though. 

S--We thought of doing it a few years ago at the University of Utah, but we never 
got around to it. I usually begin the course I’ve been teaching for the past fifteen years 
on the creative process by saying that the students may be expecting to find out how much 
they should drink in order to do their best creative work, or some wisecrack to that ef- 
fect, because the delusion is almost universal that a little help is to be got from alcohol. 
The tendency to hope for such easy approaches to creation has to be discouraged. 








C- -There is an anecdote about a young physicist on the University campus, Don 
Blazer, who recently invented a bubble chamber for detecting radiation. He said that 
he first got the idea while watching bubbles form in a glass of beer. And somebody 
said, Why don’t you use that to detect radiation? As a matter of fact he said he went 
home and tried it. 
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C--That is exactly what I had in mind. It is a fact that you load up on all the 
aspects of the problem and it defies solution, and then your attention becomes dispersed 
and involuntarily you're thinking about something else. So my question is: Is it the fact 
that the mind needed all that time to integrate, or is it the fact that you are thinking about 
a specific item that triggers off the solution? Which is it? What is going on? Is it a com- 
bination of the two, or do certain stimuli trigger off a solution? 

S--I think I can answer some of these questions, in part, in what I am going to 
say later on. What we are trying to understand is not only the clearance of the mind for 
free and full apprehension of whatever new configurations will suffice it in its search for 
insight; we are even more eager to see how these new insights come into being. As the 
remarks just now introduced by way of discussion have intimated, more is required for 
the production of new insights than merely cultivating a mood of relaxation. The extreme 
procedure followed by A. E. Housman in seeking release from the stresses of will and of 
attention must be recognized as perhaps unsatisfactory in itself and certainly as incom- 
pletely accounting for the production of fresh insights. He daily made up his mind, and 
prepared his body, to compose automatically. In the afternoons, his least intellectual 
period, after drinking beer for luncheon, he went out to walk, idly, observing the changes 
of the seasons, and as he walked along there would bubble up, apparently from the pit of 
his stomach, words, phrases, stanzas, or more rarely a complete poem. Sometimes 
the fragment so produced would be augmented spontaneously at tea time, but sometimes 
nothing whatever would evoke anything further. Relaxation was evidently not the whole 
secret. Housman's procedure has not been found to be generally useful. Merely to 
relax, to diift away in total absence of mind will ordinarily bring us only where the 
spasms of habit may lead, or to sleep itself -- or to both ends at once. An illustration 
is afforded by the story James Grier Miller tolls of a professor, who went upstairs to 
change his tie before going out to dinner and landed in bed, in the redintegrative process, 

I suppose we may call it, of taking off his tie and then, naturally, unbuttoning his shirt 
and so on. 

To clear the mind for creative action, something more positive than relaxation 
and less restrictive than voluntary diversion is required. If the mind is merely dis- 
lodged from some one of its objects it will swing to another. And there, if its attention 
is not redirected by the intrusion of new stimulation or advanced in the sequence of 
associations, it will remain until dispossessed by indifference. In this fashion it will 
go on darting or drifting to one or another configuration, obvious or recondite, each 
one of which will distribute the energies of attention in some more or less familiar way, 
in accordance with that huge complex of pattern, scheme, system, ordered arrangement 
of whatever kind in terms of which we live and move. Human experience is very persist- 
ently configurated. 

The escape from the closed world of configurations is accomplished neither by 
drifting nor by willful propulsion. The creative mind is drawn out of it by desire for 
an order that does not exist, for some distribution of its energies which no configuration 
is available to determine. To ’’think aside” is to anticipate so intensely that desired play 
of energy that all those forms which fail to effect it, that is to say all configurations, sink 
into insignificance. The mind thus disappointed of closure but alive to its unsatisfied need 
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for an act of attention which it cannot make is freed from attention to every configuration. 
What remains to it is an indeterminate plenitude, elusive excitements and multitudinous 
shifting intimations of idea and imagery . 

Jacques Maritain has written a suggestive essay on "The Dark Night of Poetry, " 
in which he remarks upon the fascination of the descent of the mind into its own wealth 
and darkness, the indeterminate stir and stillness of the mind. The paradox seems to 
express it, because there is a sense of undisturbed self-possession, at best, as well 
as of rich activity. The experience seems to be a little like a mystical experience. It 
has to do with what Whitehead calls, in a definition of mysticism, the "as yet unspoken” 
aspects of experience. In a poem about the creative process, "Long-Legged Fly, ” 

Yeats makes a refrain of the words " His mind moves upon silence . ” Silence, so im- 
portant to the mystic, implies the absence of pattern, configuration, the explicit matter 
of expression. But this is not all. The silence of the creative mind, like that of the 
mystical mind, is sustaining, not an emptiness . It is a substantial profundity of being. 
Though it comprises the play of energies that tend toward resolution in insight, they 
yield an impression less of developing meaning than of developing life . Profoundly sub- 
jective and without representation in the independent form of any configuration, these 
determinants of meanings still inchoate seem for the time being merely aspects or 
movements of the deeper personal life. That may be why the creative individual is so 
much implicated, not in a selfish way, but in a personal way, with what he is doing. In 
some degree he lives these things, as he lives for them. 

Creation is prompted and sustained by impulses and resources of energy brought 
into play by the need to transcend the life that is defined in established configurations, 
whether conscious or unconscious. Talk about art as the expression of neurosis can be 
dismissed on these grounds. Creation is not the exploitation of any fixation, hidden or 
overt. It is always to some extent an assault upon fixation, as we might gather from 
the disturbance it causes in fixed minds when it evokes anything more than indifference. 

It offends because it disturbs habits and baffles old-fashioned understanding. I think of 
Einstein’s early reception by some dubious scientists — I*ve used the word dubious in 
two senses there. And Jacques Hadamard has called to attention the fate of Evariste 
Galois, killed in a duel at the age of twenty, whose memoir illuminating higher algebra 
was rejected by the Academy of Science as unintelligible, and only fifteen years later 
was accepted by a few delighted minds, still lagging well behind Galois. Recognition 
and true appraisal of new insight can be expected only of those who share the passion 
of creative minds to move with their whole energy, in a life of larger scope than that 
determined by even the best insights already in possession. 

Our discussions of creativity, as well as a great deal of previous work on the • 
subject, have emphasized the need for a clear central concept for the understanding 
of the creative process . The old approach has been too superficially descriptive, 
representing the process as a series of various actions. I believe that looking at it as 
essentially a single action may be more fruitful than some other approaches have been. 

It may be conceived of as an exercise of the configurative powers of the whole psyche, 
involving all its substance, the play of its entire energy. If the whole psyche is involved, 
this means in some sense the body, the whole psychoplasm. William Butler Yeats, 
writing of the necessity for such inclusiveness, says in one passage that our thought 



rushes "out to the edges of our flesh" and, in another, that the poet "thinks in a marrow 
bone.” The creative action is more than cerebral, more than visceral, involving more 
than mind and heart, intellect and emotion. Yeats has expressed his sense of this full- 
ness in another of its aspects by saying that in "the one moment of creation" we are at 
the same time both asleep and awake. No doubt he meant to imply a dreamlike activation 
of the mind, as well as its ordinary conscious processes, and its still deeper preconfigu- 
rative silence. In the richest moment of the mind's action, we seem to be looking as if 
into our sleep, not merely into dream, but into the deep undesigned life. The whole sub- 
jective sphere is activated in the production of a new configuration. 

C- -Einstein fell into a depression for two weeks after he wrote up his notions on 
relativity. 

S--Yes, exactly. It took everything he had. 

No less than the full resources of our ueing and the utmost discipline of spirit 
are required for the most significant creation, that which augments the structure of hu- 
man vision by introducing some new, vital configuration of idea or form among those 
available to organize experience. The consummatory moment of creation, coming spon- 
taneously, never directly out of calculations and strivings, may seem dissociated from 
effort, a mere miracle. But, as Paul Valery declared emphatically of this attainment, 
"The spontaneous is the fruit of conquest. " Certainly the conquest includes the labor 
that initiates production, and it is completed by further labor. Yet such work, however 
exacting and exhausting, hardly in itself justifies the term conquest. There is a further 
effort which does, that which sustains the freedom and fullness of the mind's movement 
in evolution of the configurative life of the individual and his society. 

Even if the concept that I have proposed is thoroughly sound it may not be very 
useful. It affords grounds for discrimination between the supreme and essential crea- 
tivity, power to effect an evolution in the very structure of human vision, and minor 
kinds of inventiveness involving only novel use of the existing structure, as in problem 
solving. But it leaves in obscurity the actual process whereby new configurations are 
realized in the materials of a medium. Perhaps it can help in defining criteria for the 
identification of creative talent. 

According to the concept I have advanced, an idea of creation as the single action 
of the whole mind in production of some new configuration, the primary evidence of 
creative ability must be freedom and power in configurative action. These are the things 
that we should investigate and learn to recognize and to foster. Accomplished use of a 
medium or thorough knowledge of it are not in themselves indications of creative ability. 
They may be gained through devoted benchwork or prolonged reading. Media are wholly 
mastered only in the exercise of creative power in production, by efforts to shape the 
substance of the medium in new configurations. 

The creative mind is distinguished by such a need and such a love for both pre- 
configurative realization and precise and full configurative action as drive it to perform- 
ance, exacting and continual, by a desire to move, to savor its power in exercise, in 
being, not in having been. To be creative one must strive not to come to repose in any 
configurations, in certainties that have already been elaborated, not even to end in one’s 
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own favored and best insight, not to rest in the dead ends of one's own achievement, 
never to become one’s own effigy. Such arrest easily overcomes even the young. It 
is usual in the subsidence of old age and common amid the fears of middle age, among 
those who want to be sure that they are not going to have to take risks, to spend time 
on problems that may not yield results, or to turn out to have been wrong. One might 
consider as an example of energy continually reaching toward new design Picasso's 
transformation of his lithograph of a bull, in which he creates first the naturalistic 
image, then one abstraction after another, each time destroying the preceding image 
on the stone and creating still another, as if he were displaying in the endless freshness 
of a single theme the mind’s inexhaustible power, I suspect the same sort of energy 
would distinguish the highly creative scientist. 



DEMOGRAPHIC CULTURAL AND PERSONALITY ATTRIBUTES OF SCIENTISTS 



Robert H. Knapp 
Wesleyan University 



S- -I think the proper thing for me to talk about here is not creativity, though 
this subject holds keen interests for me, but rather the character and characteristics 
of scientists. In the past several years I have been engaged in four separate studies 
bearing upon the recruitment, education, and personal characteristics of scientists. 

But before discussing these four studies, I should like to comment that I find little 
correspondence between the characteristics of scientists as I discern them, and the 
quantities of the "creative personality" as they have unfolded in our preceding discus - 
sion. It may well prove that the "creative" scientist is a rather special type of scien- 
tist in background and personality attributes — that creativity in science requires an 
especially fortunate combination of qualities only rarely associated in a single individual . 

The four studies dealing with scientists are the following: First, there is a study 
of the undergraduate origins of American men of science holding a Ph. D. and taking 
their undergraduate degree between 1924 and 1934. This study (Knapp and Goodrich), 
also included some case studies of colleges in an effort to determine factors related 
to high productivity of such scientists. The second study (Knapp and Greenbaum), which 
is entirely statistical, sought to determine the undergraduate origins of younger Ameri- 
can scholars in the years 1946-51 in all fields of learning. Here again analysis of the 
data was made in terms of the characteristics of institutions producing a high "yield" 
of scholars including potential scientists. The third study (Knapp), involved an examina- 
tion of the student body at Wesleyan University to determine the demographic and psycho- 
logical attributes which differentiated science majors from those in other departments 
of the curriculum. Finally and fourth, should be mentioned a factor analytic study 
(Knapp, et al. ), of the temperamental attributes of eminent men, employing data com- 
piled by the late Dr. Edward L. Thorndike and including data on eminent scientists as 
well as men of other orders of distinction. 

Let us consider the first study. Here we set ourselves a task of finding out the 
undergraduate origins of Ph. D. ’s in American men of science who took their Bachelor's 
degree between 1924 and 1934, this period being selected as a fair time sample between 
the First and Second World Wars. Having determined the number of Ph. D. 's in science 
produced by all American colleges, universities and technical schools during this period, 
we reduced this number to a rate of production. This rate was based upon the total 
number of their male graduates during these years and thus represented a sort of "yield 
per acre" measure in which the advantage of larger institutions was eliminated. This 
index was then subjected to a further analysis and broken down into the yield for each 
of the several principal departments of science. Of course, these measures were of 
little value in smaller institutions, but in larger universities they served to indicate 
the relative achievements of different scientific departments within an institution. 

In our preceding discussion we have several times raised the question of whether 
scientists may be considered as a relatively homogeneous body of men or whether on the 
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other hand they do not constitute a very wide and diverse variety of personality types, 
especially as we move from one field of science to another. I am going to take the po- 
sition here that although there are undoubtedly differences between scientists in different 
fields of endeavor, nevertheless there are probably certain abiding and common charac- 
teristics that warrant our treating them as a single population, at least in preliminary 
studies. Support for this view is to be found, I believe, in our finding that the :~-hieve- 
ments of departments within a single university tended to be reasonably highly corre- 
lated, There appeared to be a general factor determining the output of scientists from 
an institution with relatively small variation from department to department. 

We pursued this problem a little further. Selecting approximately 50 large uni- 
versities for study, we inter cor related their Indices of scientific production between the 
several fields of science under consideration. The resulting matrix was then subjected 
to a factor analysis in order to discern the clustering of these several sciences . From 
this analysis 3 factors emerged. The first was saturated very highly with Mathematics, 
Physics, and Geology, moderately in Chemistry, and very slightly in Biology and Psychol- 
ogy. This factor seemed to define a physical and mathematical emphasis. Our second 
factor showed very small saturation in Physics and Mathematics but a conspicuously high 
saturation in Biology and Psychology. It seems clear that here the emphasis upon life 
processes is a strong determinant. Our third factor showed conspicuously high satura- 
tions in Chemistry and Biology, a moderate saturation in Physics, and was markedly low 
in Mathematics, Geology, and Psychology. This third factor seems to saturate in those 
fields which have a strong and formalized laboratory emphasis. I think this factor anal- 
ysis may be of some value in revealing the community of interests between different 
scientific disciplines . 

From the above digression I would like to return again to the overall index which 
we computed for all American universities, colleges, and technical schools. Having 
satisfied ourselves that this was a reasonably homogeneous measure, we now chose to 
regard this as an independent variable and to determine how it might be correlated with 
certain independent factors. For this purpose we selected two samples for intensive 
study from the total body of institutions considered. The first sample consisted of ap- 
proximately 150 liberal arts colleges, the second of about 90 universities. We next 
correlated a considerable variety of independent variables with our indices of production 
in both samples. These included such things as entrance requirements, student -faculty 
ratios, minimum costs of attendance at the institution, geographical location, size, etc. 
Two of these variables emerged of over-riding importance in both samples. The first 
of these was the geographical gradient showing the Middle and Far West to be very high 
in the production of scientists, the Eastern seaboard of moderate production, and the 
South conspicuously low. We sought to relate this geographical gradient to a number of 
economic and population indices without much success and were eventually forced to think 
that regional cultural differences were primarily responsible. 

The second consistent and important factor, evidenced even after the geographical 
gradient had been partialled out, was the minimum cost of attendance at the institution. 
This figure was taken directly from American Universities and colleges and showed a 
very wide variation. But the correlation between our index for scientific production and 
the minimum ccst of attendance was not linear. Rather, it showed a distinct parabolic 
character, with institutions in the middle cost being conspicuously higher in the produc- 
tion of scientists than those of either very high or very low cost. Thus, Harvard, Yale, 
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Princeton, and other big-name Eastern universities and colleges were almost uniformly 
low in the output of scientists, while, on the other hand, there were many very obscure 
and impecunious institutions of excessively low cost that were equally low in output of 
scientists. 

The second stage of this study involved a case study of a number of American 
colleges, initially totaling 25, which were selected because of their special success in 
the production of American scientists . Whereas our statistical study was confined to 
scientists graduating between 1924 and 1934, our case studies sought to encompass a 
larger historical perspective, extending into the 19th century. When our data were 
finally complete, it seemed to us that we could clearly detect a general pattern of evo- 
lutionary development in the small liberal arts colleges that we were examining. Vir- 
tually all were initially founded as educational organs with some sectarian denomination. 
In the beginning they were engaged primarily in the training of ministers, teachers, and 
other persons in social service. Then came a period during which they seemed to turn 
from a sectarian to a secular dedication, and this development is coincident with the 
establishment of courses in science, scientific laboratories, and the appearance of 
scientists from among the ranks of their graduates . We found it appropriate to designate 
this development as the period of first secularization. In a great number of cases this 
period proved to be the college’s productive period in the graduation of future scientists. 
In a number of institutions a second state of secularization succeeded the first, and in 
this period the institution characteristically began to send an increasing number of its 
graduates into the professions of law, medicine, banking, and industrial management. 

In several cases this second stage of secularization showed a marked falling off in the 
output of scientists. The sequence might be described quintessential^ as follows: 
first, religious and service dedication; second, scientific and technological interest; 
third, professional and managerial emphasis. It might be noted, too, that certain 
Protestant denominations moved from sectarian to secular commitments earlier than 
others. Thus, Congregational schools generally effected this transition around the turn 
of the century, Baptist schools and Methodist schools somewhat later, and Lutheran 
schools still later. 

I should like to digress a moment from this observation on historical evolution 
of liberal arts colleges in America to some comments that bear upon the psychodynamic 
attributes of the scientific personality. It would appear that scientific interest has its 
strongest hold in a transition period between the acceptance of orthodox Protestantism 
and the acceptance of a more loose secular world outlook. If I understand this phenome- 
non correctly, science is an interest which emerges to the forefront following the re - 
linquishment of Protestant orthodoxy and before the development of a fully secularized 
outlook. This latter stage of development appears to be associated with achievement 
striving directed towards the manipulation and exploitation of human materials and does 
not encourage scientific interests. It is a point of view that no longer places a great 
premium upon the value of impulse control and the employment of repression as a pri- 
mary and preferred defense mechanism within the personality. It appears to me that 
the first stage of secularization retains the ideals of impulse control while combining 
them with a secular preoccupation with the physical world. It may well be that one of 
our difficulties in the recruitment of scientists at the present time lies in the propensity 
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of our culture to place a premium upon expression and competency in human relations 
rather than on impulse control and concern with the structure of physical things. The 
history of the rise and decline in the production of scientists from some of our older 
Eastern colleges seems to bear out some general interpretations of this sort. It was 
a conspicuous finding, for example, that the number of science majors in New England 
colleges which had long been secularized was markedly lower than that in the Middle 
Western and Far Western "grass roots" liberal arts colleges. 

’There are a number of other aspects of this case study which might be developed 
here, but which, excepting for 2, I will forego at this time. I should like to report on 
one of our efforts to identify the qualities of productive science departments and the 
attributes of teachers who had enjoyed notable success in leading their students into 
scientific careers . We were able to examine the correlations between the output of 
science departments and certain conditions prevailing within them respecting curricular 
features, departmental morale, physical facilities, etc. Two of these factors showed 
significant linear correlation with their output of scientists, namely, the "espirit de 
corps" of the department as manifest both in the warmth of human relations and contacts, 
and the severity of academic standards. One emerges with a picture of the successful 
department as being characterized by a warm but demanding intellectual environment. 
Material aspects of the department, including libraries, seem of much less importance 
and frequently showed a parabolic relation to productiveness. Possibly this parabolic 
relation is an artifact, for the most superior material resources were commonly found 
in higher cost institutions which had relatively few majors in science and recruited 
their students from middle and upper middle class constituencies among whom the pro- 
fessional and managerial callings, not science, seemed to have strong appeal. 

Second, we were able to study the personal attributes of teachers, and to identify 
those which distinguished successful from relatively unsuccessful teachers (bearing in 
mind that we are speaking of their success in inducing their students to enter a scien- 
tific career). Here we were confined to 2 samples, one of biology teachers and one of 
chemistry teachers. We took those attributes which correlate significantly with teacher 
success and then submitted them to factor analysis. We emerged with 3 factors in the 
case of both samples, though for the chemists the third factor was only suggested. The 
first factor, which gave us the highest loadings in both samples, we called masterfiilness, 
and comprised such traits as demandingness in standards of performance, assertiveness, 
and entrepreneurial vigor. The second factor might be called warmth, and showed high 
loadings on such traits as breadth of interest, administrative participation, teaching zest, 
etc. Finally, our third factor showed high loadings, at least among the biologists, on 
such things as intellectual eminence --i.e. , research output --and personal dignity. 

These ratings were in good part derived from questionnaires addressed to the former 
students of these teachers who had become scientists themselves. 

Our picture of the successful teacher, then, sums up to this: There are appar- 
ently , so far as our data may be trusted, 3 general constellations in the characteristics 
attributed to superior teachers: masterfulness, warmth, and intellectual eminence. 

These constellations correlate with the criterion measure in declining order. Presum- 
ably a teacher might achieve a distinguished record by possessing only two of these three, 
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or possibly only one. From my experience in these case studies, however, I cannot 
recall a single teacher who attained signal success without demonstrating a significant 
measure of at least the first two. Most of these effective teachers ultimately might be 
described as fitting the image of the masterful and demanding but warm and interested 
father . Many of them had qualities which it seemed to me might well have lead them 
to considerable success in entrepreneurial undertakings generally, especially the dis- 
tinguished Chemistry teachers. 



I would like now to turn briefly to our second statistical study of American 
colleges and universities. This study, as I indicated earlier, was confined to an exam- 
ination of college graduates between the years 1946-51. We obtained a roster of prom- 
ising undergraduates for these years, i.e. , those who had won fellowships, scholarships, 
and other such distinctions, and with this as a base proceeded to compute indices of pro- 
duction for all American colleges and universities in the same manner as before. This 
time, however, we dealt with all fields of graduate scholarship (excepting law and medi- 
cine) and thus evolved indices for social studies and humanities as well as science. With 
respect to the recruitment of scientists, the findings of our earlier study are pretty well 
sustained for this later period. Thus, the differences in geographical region, between 
types of institutions, etc. hold up. But in one respect there is a very radical difference, 
namely, that of the cost of attendance. We find that our high cost institutions, which 
previously were inferior in the production of scientists, are the most productive. Harvard, 
Yale, and Princeton are notable examples of this shift. 

Now, it appeared to us that this shift probably could be explained by considering 
tile student clientele of these institutions during the "G. I. epoch. " During this period the 
superior American student was extremely mobile, and he gravitated, aided by federal 
subsidy and other financial assistance, to these prestige fill institutions. Thus, students 
of keen scientific interest?, frequently coming from "grass roots" origins, found their 
way in large numbers into these high cost institutions with the result that they could ex- 
ploit the faculty and facilities made available there. Previously, in the twenties and 
thirties, these conditions did not prevail, and entrance to these high cost institutions 
more or less required privileged economic standing. Among students from upper eco- 
nomic groups, as I have sought to show elsewhere, the disposition to enter science is 
probably quite low for various reasons. 

It might be worthwhile to comment briefly on some of our findings in this study 
respecting the recruitment of younger scholars in the social sciences and humanities. 

Our earlier study of the origins of American scientists showed for example, that both 
Southern and Catholic institutions were low in the production of scientists . We had specu- 
lated that they might be correspondingly high in the humanities, but our second survey 
did not show this to be the case. Rather it demonstrated that these two classes of insti- 
tutions were relatively more unproductive in the humanities than in the sciences . If we 
ask where our younger students in the humanities are coming from, the answer is une- 
quivocal. In very great proportion they come from the high cost private universities 
and colleges of the Northeast section of the country. Compared with the sciences, their 
recruitment is highly confined, perhaps unwholesomely so, to a distinct class of insti- 
tutions located in a confined segment of the nation. The social sciences show a greater 
degree of dispersion in recruitment pattern, with less concentration in institutions 
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of high cost and less concentration in the Eastern states . The natural scientists, as 
we have suggested already, are the most widely and evenly recruited and seem to draw 
upon our ’’educational grass roots” in a manner distinct from those in humanities and 
the social sciences. 

The third study I would like to discuss was done for the Office of Naval Research 
and really had 2 parts; first, the examination of demographic and sociological variables 
correlated with the selection of each of our three divisions of the curriculum, i.e., 
sciences, social studies, and humanities; and second, a personality study of a selected 
group of superior juniors and seniors in these 3 divisions employing largely projective 
tests. With respect to the first, I shall skip over a number of facets of the inquiry of 
purely local interest and devote myself largely to the socio-economic correlates of cur- 
ricular commitment. Here we emphasized our graduating classes from 1945-51, ex- 
cluding those below a B-average academic performance. 

Our earlier studies of the origins of American scientists had led us to expect 
that scientists would tend to be recruited from humbler social origins than scholars 
in other disciplines. We therefore combined all Wesleyan graduates from the classes 
of 1945 -*51 inclusive and sought to determine how the socio-economic station of their 
parents, which was a matter of record, might be related to the field of concentration 
which they selected in college. Our expectations were nicely confirmed. We found a 
third of our scientists coming from families whose livelihood was earned in non-white- 
collar occupations and another 60% coming from essentially lower middle class homes. 

Only 9% came from upper middle or higher class families. It should be mentioned in 
passing that we did not consider pre -medics as ’’science majors, ” since they appeared 
to us to have quite different motivations and dedications. Actually, they were drawn 
much more frequently from higher economic origins. 

By way of contrast, students graduating in social studies tended to be drawn from 
upper middle class families and the families of professional men. Our humanists, in- 
cluding those in the arts and in literature, showed an interesting pattern, being drawn 
from the socio-economic extremes. Thus, it appeared that we had one group of ’’cavalier” 
aesthetes and another group of ’’proletarian” aesthetes. 

A second feature of this particular analysis might be dealt with here, namely the 
type of community from which our scientists were drawn . We have earlier seen that the 
grass roots Protestant college of the Middle West, drawing its students from non -metro- 
politan centers, proved highly productive of scientists. Would our own scientists like- 
wise be drawn from small towns and rural regions? After some exploration, we were 
able to demonstrate that there were indeed trends in this direction. We classified all 
students living within a 30 -mile radius around Philadelphia, New York, and Boston and 
compared these with the rest of our graduates. We then discovered that only 30% of 
metropolitan students elect science, while 43% elect social studies and 49% humanities. 

Our grass roots hypothesis respecting the origins of scientists seems - - at least logically - - 
sustained by these data. 

I am going to skip over most of the further findings from this statistical analysis, 
noting in passing only that students from public high school showed, as might be expected, 



a greater inclination to science than those from preparatory school, that I.Q. did not 
appear to vary significantly, and that religious differences, though expected, did not 
materialize . 

I should like now to pass on to the second phase of our study of the Wesleyan 
student body, that devoted to an examination of personality differences between majors 
in science and majors in other departments of the curriculum. Here we employed 
rather carefully selected samples of Wesleyan seniors, all of whom showed an academic 
average of B or better in their major field. Between oO % and 60% of our seniors continue 
to graduate studies of one sort or another, and though we are admittedly dealing with 
undergraduates, there is good reason to believe that they are not unrepresentative of 
more advanced scholars in their field. We employed 3 projective tests as our instru- 
ments: first, a group Rorschach; second, the Blackie Test as devised by Gerald Blum; 
and finally, a group Thematic Apperception Test as devised by David C. McClelland. 

The results of the group Rorschach were in the main disappointing to us, despite 
the fact that we made some quite exhaustive efforts to find variables or combinations of 
variables which would correlate with curricular commitment. The only result which 
should be noted here was the propensity of our students of science to yield a significantly 
higher ratio of Sum C to M than students in the other two divisions. This merely confirms 
the expected extrotensive orientation of these students . In contrast, students in literature 
and the humanities were noteworthy for an opposite trend with respect to this ratio. 

The second projective test employed in this battery was the Blackie Test of Gerald 
Blum. Work with this instrument was largely carried out by Dr. Richard Teevan, who at 
that time was completing his undergraduate distinction v/ork in the Wesleyan Psychology 
department. As you know, the purpose of the Blackie Test is to identify fixation at different 
Freudian psycho -sexual levels, or sensitivity to typical traumatic experiences of child- 
hood. While the results showed marked oral fixation among students of humanities, and 
also certain distinguishing patterns of fixation among those in social studies, the students 
of science were almost universally the least disturbed on all variables. While a fuller 
interpretation will be left to a more appropriate section, 3 hypotheses may be advanced 
here by way of explanation. First, it is possible that as a group the scientists possess 
actually less disturbance in these areas of childhood trauma. Second, it may be that the 
nature of the test, dealing with an animal, invited an attitude jf scientific dispassion, 
lowering their emotional responses to the pictures . Third, it may be that scientists as 
a group are not characterologically given to identification and the projection of affect, 
being rather disposed to deal with affect by repression. 

The third projective test which we employed consisted of the Group Thematic 
Apperception Test as devised and administered by Dr. McClelland. This test was given 
to the same selected group of academically superior majors in the 3 departments of our 
curriculum. In scoring the resulting protocols, we employed eight variables, the selec- 
tion of which was dictated by hypotheses arising from our previous studies of the origins 
and characteristics of scientists. These were: dramatic saliency, heterosexual reference, 
aggressive thema, guilt reactions, pattern of conflict solution, reaction to father or father 
surrogate, reaction to mother or mother surrogate, and incidence of thema dealing with 
vindication or debasement. 
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On the dramatic saliency variable as noted by our judges, the scientists are 
conspicuously and significantly lower than the other two, while the students of humanities 
are the highest. This variable which defined the general quality of dramatic impact and 
vigor appeared to be lower among die scientists fpr at least 3 reasons: the lack of ag- 
gression, guilt or vindication in the stories, and a noticeable reluctance to \ ig the 
story to a decisive and clear conclusion. I shall discuss these qualities later, although 
they seem to me a highly significant aspect of the thought processes of this sample. If 
we regard dramatic saliency as a measure of creativity, then we must conclude that 
scientists are quite uncreative in the medium of imagining and contriving structured 
human relations. This sustains the conclusions of other studies that scientists as a 
group are but poorly at home in this domain of experience, however creative they may 
be in other non -personal areas. I »2elieve r however, as has been proposed, that some 
other type of T. A. T. employing impersonal or mechanical things might well yield rich 
and structured protocols from our scientists. 

On both the measures of sexual reference and guilt reactions the scientists have 
the lowest averages, though they do not attain to secure levels of significance. On ag- 
gressive thema, on the other hand, the scientists are clearly the lowest while students 
of the humanities are the highest. This difference far exceeds the one percent level of 
confidence. It clearly suggests the conclusion contained in our fourth study which I have 
yet to report, that interpersonal aggression is an area of great sensitivity for the scientist, 
arid that it may be possible that his flight from the domain of human preoccupations is mo- 
tivated by his special discomfort with this order of personal exchange. 

Probably the most significant finding to emerge from this body of data, in my 
opinion, concerned the solution pattern provided by the thjree groups of students. In the 
first place, we find that the scientists yield far fewer solutions to their stories which 
might be called "overt"; that is, which involve a clear change of situation as a result of 
actions undertaken. Second, when we examine those stories to which there is no "overt" 
solution given, we find that our scientists are very prone to provide solutions of a "repres - 
sive" character. Here conflict is engendered, but it leads to no action. Rather the con- 
clusion is presented that the individuals in question merely "cool off" or "forget about it. " 
Finally, we may note that the scientists significantly more often provide no solution what- 
ever or several alternatives from which the reader may elect. All of this adds to a pic- 
ture of sensitivity in the area of interpersonal conflict combined with a propensity to em- 
ploy repression or isolation in dealing with the affect engendered by such conflict. 

On our sixth variable, we were able to demonstrate that the scientists were sig- 
nificantly lowest in expressions of hostility to the father or father surrogate, and at the 
same time they were also lowest (though not significantly so) in the introduction of a be- 
nign father figure into their stories. In passing we might note that the very high incidence 
of hostile father figures was a marked characteristic of students in the humanities . On 
our seventh variable, dealing with references to the mother figure or surrogates thereof, 
we could determine no significant differences among our three groups of students. 

Finally, we attempted a measure which described the involvement of the principal 
character of the story in either debasement or a vindication thema. Here we found a clear 
tendency among the scientists to avoid the extremes of both as though moral evaluation of 
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the protagonist’s fate was a distasteful consideration. As you might now surmise, stories 
of marked debasement or vindication were typically and abundantly produced by students 
in the humanities, and somewhat less by students in the social sciences. 

In summary, therefore, we found that 4 of our 8 variables discriminated signifi- 
cantly between our scientists and our controls. The protocols of our students of science 
proved to be characterized by low dramatic saliency, low incidence of aggressive thema, 
paucity of over solutions, and marked tendency to "repressive" or evasive solutions, low 
incidence of hostility to father figures, and the avoidance of debasement or vindication 
thema. This general picture is quite consonant with Dr. Barron's observation that the 
protocols of scientists tend to be high in description and low in personal involvement . 

I believe, taken with the other evidence on hand, they suggest that the scientist is given 
to a clear "distantiation" in dealing with and considering human relations and, as I have 
noted previously, is disposed to employ repression and isolation in coping with affect 
arising from human exchange. 

Now, I should like to mention very briefly the fourth of our studies. This con- 
sisted of a factor analysis of temperament ratings of some 90 men of great eminence 
undertaken by the late Dr. Edward L. Thorndike. These 90 men were drawn from the 
past two centuries of our history and included representatives of 5 general groupings, 
namely, writers, military figures, scientists, entrepreneurs, and statesmen. Dr. 
Thorndike passed away before he was able to complete the analysis of the forty-odd traits 
on which ratings had been made for each of these and therefore we undertook to factor 
analyze the data and to compare the several groupings. The results were most inter- 
esting in many respects, though I shall mention here only the findings on the third factor. 
This factor defined essentially a disposition to participate in and even enjoy controversy, 
debate, and such forms of interpersonal aggression. High in this, naturally, were the 
politicians and statesmen, but conspicuously low were the scientists. We have, then, 
further supporting evidence that interpersonal aggression is a source of particular sen- 
sitivity and distaste to scientists, though I should add that the sample is limited, num- 
bering only 7 of the 90. With respect to the other 3 factors, the sciet list group, which 
deviated from the mean was not conspicuously deviant as compared with other groups. 



SOME POSSIBLE RELATIONS BETWEEN 
EXPRESSION ABILITIES AND CREATIVE ABILITIES 



Calvin W. Taylor 
University of Utah 



S — It seems appropriate at this time to restate a few comments made in my 
brief greeting at the beginning of the conference. A first comment is that the confer- 
ence topic was chosen very carefully. We are trying at this early stage of knowledge 
to determine (1) the nature of the creative type of scientific talent, (2) the best methods 
now available for identifying this type of talent, and (3) the research that can be done 
in the future so we can better identify creative scientific talent. 

According to some persons, all that is really needed for progress in science is ' 
the combination of a good problem, a good setting, and a good man. We believe that 
we have a good topic for a conference, a good setting at this mountain lodge, and we 
know that we have an excellent group of participants. We are happy to have all of you 
here with us and appreciate your many contributions throughout the conference. 

I also believe that it is appropriate that this conference is being staged in Utah. 
You may recall that Edward L. Thorndike reported in Scientific Monthly on the geo- 
graphical origins of scientists listed in the 1938 American Men of Science and found 
that Utah surpassed all states by a sizeable margin. In fact, there were 11.7 scientists 
born in Utah per 10, 000 white population (1890 census) compared to 8.0 for the second 
ranking state and 4.7 for the median state. Richard T. Wootton, who is working on a 
dissertation in educational administration on our campus, recently reported to me that 
he had repeated Thorndike’s study based on a later edition (1949) of American Men of 
Science and found that Utah had maintained its strong lead over all other states. We 
recognize that most of the Utah -born scientists have moved to other regions in the 
country where they have made their main contributions in science. Consequently, we 
feel that it is only fair for some scientific activities, such as this conference, to come 
to Utah in return for the scientific manpower furnished by this state to the rest of the 
nation. We also ponder how much less our nation might currently be challenged in 
scientific manpower totals if many other states had been able to attain Utah's propor- 
tionate level in scientific manpower production . 

During the past several years I have been working periodically on verbal fluency 
and other expressional abilities. We now have a two-year study in progress, with Air 
Force support, on the measurement of communication abilities on which we are fortu- 
nate to have Brewster Ghiselin of English, Boyd Sheets of Speech, and others working 
with us. Most of us on the current project have convictions that many expression types 
of communication scores will later prove to be interrelated in several ways with crea- ' 
tive talent scores. For example, a person must show at least some minimum produc- 
tivity to be creative and we are obviously studying productivity when we study expres- ' 
sion abilities. It is quite possible, however, that the relationships between expression 
abilities and creative abilities will not necessarily be very simple. Instead of reporting 
on our current research in its early stage of development, it seems more appropriate 



213 




to speculate about some possible relations between expression abilities and creative 
abilities. Most statements about suspected relationships are highly tentative at this 
early state of knowledge; nonetheless, it is hoped that these statements will prove to 
be provocative. 

One expression ability that has reappeared in several factor studies is called 
ideational fluency . The scoring of the landmark tests for this factor stresses sheer 
quantity of verbal output. In these tests the subject is told to write any idea that comes 
to mind about a given, vague topic, whether his ideas seem trivial or not. I have often 
thought that at least some of the extremely high scorers on ideational fluency may in 
their test performance be approaching the manic's wild flight of ideas which show a 
rapidly deviating train of thought, flowing almost unguided from one tangent to another. 
There is some doubt that such persons are very capable of identifying which one or two 
of their voluminous number of ideas are best and thus worth singling out for some par- 
ticular purpose. At the other extreme from the person with the mass production of 
ideas is the person who has never produced, has never really expressed himself much, 
and therefore has not yet demonstrated any creativity. In seeking creative talent, per- 
haps we are interested in those who can have fluent bursts of ideas if at the same time 
they can validly identify the best of their own ideas. 

In a discussion with Anne Roe, I received the impression that many of her 
eminent scientists needed only to have two or three high quality ideas for development 
each year in order to add other good links on their usually continuous chain of research 
studies. To maintain sustained production in science, a scientist may need only one new 
good idea to pursue within a reasonable period of time after he finishes a previous piece 
of work. Thus, we may be searching for persons who do not necessarily produce a 
large quantity of responses on expression tests, but who produce high quality responses 
or a relatively high per cent of creative responses. We might increase our chances of 
identifying such persons if we would extend the time limits or deadlines considerably on 
our tests so that the creative process will have a greater possibility of operating in some 
examinees to yield truly creative responses . We have thought that some existing tests 
might be modified to become "take-home” tests which would permit the examinee to add 
respqnses whenever he gets relevant ideas during an extended period of a week or two. 

As implied above, a person who scores high on ideational fluency may not neces- 
sarily be the type of person we are seeking. Instead, there may be a curvilinear rela- 
tionship between ideational fluency and creative ability in science. It is possible that 
there is an optimum amount of fluency, with the best prospect being the person who gives 
you a middle-of-the-range total number of responses, which responses for the most part 
are high quality or even creative ones. However, if two persons are about equal in their 
ratio of high quality ideas produced, the more idea -fluent person should be the better 
prospect. 

It seems that creative ability should be more related to quality than to quantity 
of expression. To date, however, not many of the expression tests have been designed 
to measure quality of production nor have there been nearly enough attempts to develop 
explicit quality scoring systems for expression tests. 
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Expression tests are often susceptible to multiple scoring which might get at im- 
portant features of the performance that would otherwise be missed by a single score. 

Some of these multiple scores might emphasize quantity whereas others might stress 
various aspects of quality. An example of multiple scores per test is illustrated in an 
expression test in which five scores could be. obtained on each person s performance: 
total number of words produced plus accuracy, originality, evocativeness, and sound 
characteristics of the responses . 

In contemplating quality and quantity of expression it may be fruitful to recall 
that some British researchers have written about the functioning of critical -mindedness 
in fluency tasks. The usual notion is that the degree of critical mindedness will be re- 
lated inversely to the degree of fluency, both across persons as well as in variations 
within an individual. This process of weighing one’s own thoughts and words prior to 
expression may differ in several ways psychologically from the process of making an 
evaluative judgment of ideas or things produced by others and presented by an examiner 
or experimenter . 

Criticism of one’s own products may occur, of course, before as well as after 
expression. Critical mindedness at the thought level may be. somewhat different from 
critical mindedness after expression. For example, the former content may not be 
nearly as explicit and the former process probably deals with more material in a less 
crystallized state since some aspects of one’s thoughts are probably not expressed 
and thus are not directly available to criticism in the expressed product. In many 
cases, the degree to which a person has committed himself to others in his expressed 
statements may not only influence his ability to be effectively critical thereafter about 
these statements, but may also restrict his ability to think flexibly and freely on this 
topic. There would be no social involvement of this type, however, in certain expressed 
statements, as in the case of a rough draft of written materials which no one else has 
seen. 

One might attempt to get either direct or indirect measures of critical -mindedness 
to see in what ways it is related to the general quality and perhaps the originality of ex- 
pressed materials. There are 2 aspects of critical -mindedness that might be of interest: 
the typical level of critical mindedness and the flexibility of critical -mindedness. The 
person who typically has a high degree of critical -mindedness would tend to inhibit and 
thus retain within himself many potential expressions that do not come up to his minimum 
level. In contrast, a person with a low level would allow a higher percent of comparable, 
potential responses to be expressed. One might attempt to measure these effects in ex- 
pression activities with some expectation that the degree of critical -mindedness may be 
related in some way to creative ability . It may be that the best combination for a person 
to be creative is to have a high profile of scores both in fluency tasks and in entirely dif- 
ferent quality of expression tasks. In other words, the best prospect may be a person 
who can produce sheer quantity of ideas upon command and who also has sufficient flexi- 
bility of critical -mindedness to produce only high quality responses in a different test 
with that emphasis. 

Tests which somehow show a person’s natural tendency - - whether he customarily 
stresses quantity of output or high critical mindedness in an unstructured task - -may 
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provide valuable indicators for use in identifying creative talent. 

When one is most creative, his ideas are often forming and flowing more freely 
than is usual for him. Contrarily, one can be so critical inwardly that there is little 
outward expression of one’s own ideas. During the creative period, the creator has 
to have some tolerance and hope for his own ideas so that at least some ideas are al- 
lowed to be expressed immediately upon formulation or perhaps somewhat later. We 
may eventually discover that a good prospect for creative scientific work is a person 
witty a moderately high, optimum degree of critical mindedness who shows little flexi- 
bility and variation below this level and who retains and rewords his ideas until they 
meet his own minimum quality standards for final crystallization and expression: 

In studying communication processes one is struck by the wide differences 
among people in their tendency to rework their communications. It seems that some 
persons have rarely ever read what they have written, let alone revising and rework- 
ing their copy many times and many ways . Of course, some of this revision, restating, 
and reworking can and does occur at the thought level before anything is overtly expressed. 
This difference in tendency to try to improve upon one’s own ideas or statements might 
well be related to the ability in science to improve the existing ideas and statements of 
ideas. Since contributors in science include those who either make minor revisions in 
existing concepts or who produce fresh concepts to replace the existing ones, it is pos- 
sible that revision abilities plus reworking tendencies either of one's own work or the a 
work of others may be related to, and actually be an important part of, creative scien- 
tific ability. There may, of course, be a difference between the tendency to rework 
the ideas, statements, or studies of others and the tendency to rework one’s own ideas, 
statements, or studies. 

Among the variety of factors that have been found in the testing of verbal expres- 
sion is a factor that could be called by several different titles such as verbal versatility, 
restating flexibility, or flexibility of written expression. This factor may be related to 
the above reworking activities in science. The nature of this factor is illustrated by a 
test in which the subject is asked to restate essentially the same given idea in as many 
different sentences as possible. 

Mich emphasis in expression testing and in writing assignments in English courses 
has been on the expansion or fluency type of task. These elaboration activities may vary 
in their psychological nature according to the amount of instructions given plus the restric - 
tions placed upon the elaboration task. The opposite requirement of boiling down ideas 
and materials, as illustrated by activities such as extracting, summarizing, abstracting, 
outlining, revising by deleting redundancies, making expressions as compact as possible, 
etc., may provide fruitful measures of scientific talent. In fact, these measures may 
strike right at the heart of the' scientific process in which compact generalizations are 
produced that efficiently encompass a much larger mass of detailed data in the area 
studied. 

Expression may also be related to creativity in terms of the communication of 
ideas between workers in the field. One person may serve as a catalyst to others as 
he expresses his ideas and has exchanges with them which prove to be provocative. 
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{ Similarly, a person may be able to communicate with other persons in such a way that 

1 the y become useful in stimulating his own thinking. It should be possible to measure 

l these valuable communication abilities. 
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[ Although some scientists can express themselves quite readily and thus appear 

\ 10 be g° od communicators for science, they do not always prove to be good salesmen 

• • ^ or building a better science. In fact, they may at times h a ve somewhat the opposite 

\ effect. Some teachers who might appear on a superficial basis to give fine sales talks 

! ^ or science may not actually be the ones who somehow communicate in a way that ignites 

\ lasting sparks of interest in many of their students. 

* 

I bi a similar vein, I recall a particular person who was in a position to advise 

| top management on scientific matters. He readily gave them answers on their prob~ 

i lems even though his advice was not necessarily based upon research findings on prob- 

I lems very closely related to theirs . We doubted that he ever really advised them to 

j have research done directly on their own problems if they wanted the best answers pos- 

[ sible. Since top management was really interested in answers - -not in research - - they 

\ received his prompt and "authoritative" advice just as readily as if his answers had 

\ been forthcoming directly from research findings on their problems. Because of his 

\ communication abilities and methods, he failed to create scientific research projects 

l *b at were often badly needed. We wondered what differences might have occurred if he 

I had been less willing and able to express ready answers, but instead would have weighed 

t bis words and hesitated to give advice until the organization's research arm had been 

j given the assignment to work directly on the problems raised by top management. 

[ Perhaps the most obvious relationship between scientific abilities and communi- 

\ cation abilities is that ultimately the scientist has the responsibility of reporting his 

study and findings to the' scientific community . This can be done by oral reports to 
audiences and by more formal published reports. Scientists undoubtedly vary in ability 
to produce clear, effective reports of either type. For instance, some may present 
such ambiguous or incomplete reports that it would be difficult for anyone else to repeat 
the study. They may also differ in how well they portray to students of science the 
actual development of their ideas and research activities from their first real hunches 
on the problem studied. They can differ in other reporting distortions which can mis- 
lead the audience. Better measures of the ability to make scientific reports should im- 
prove our efficiency in identifying scientific talent. 

It might be expected that those v/ho do their thinking essentially in terms of 
words have less of an expressional problem when they must give a verbal report than 
those who think graphically and spatially, or in other non-verbal symbols, or in images, 
or in imageless thought. Similarly, if the report should be required primarily in graphic 
form, those who do their thinking largely in these terms should have less of a conver- 
sion and expression problem than other persons. 

, During my closing comments it seems wise to reiterate that in referring to ex- 

pression abilities, we are not restricting ourselves solely to the relatively superficial 
motor responses in expression. Perhaps there would be less chance for this misinter- 
pretation if we were to use the term, "communication abilities, " which clearly implies 

217 



BBSS 



O 

ERIC 






that central as well as peripheral psychological processes are involved when a person 
receives and stores ideas and their expressions, and then formulates and transmits 
ideas to others. Thus, I really have been speaking primarily about the cruciai forma- 
tive as well as the response processes in expreasional types of communication abilities. 
Research on expression abilities certainly provides another potentially fruitful avenue 
into the difficult problem of understanding and identifying creative scientific talent. 



SOME COMMENTS ON CREATIVITY AND MENTAL HEALTH 



Raymond Gould 

National Institute of Mental Health 



S--This conference has been very stimulating for me. A long time ago as an 
undergraduate in one of the Eastern universities that has been discussed here, I was 
majoring in English and developed a strong interest in creativity. I was able to do a 
couple of small library research jobs around the topic, and it has been a growing inter- 
est since then, even though I became a sociologist and am no longer supposed to know 
anything about such things. At the National Institute of Mental Health some of us are 
quite interested in this problem. We feel that creativity has a relationship to mental 
health. Different ones of us have different ideas about this, but some of us assume 
that many people have at least a small bit of creativity or the tendency to create or to 
produce new ideas, and that insofar as they are able to give satisfying expression to 
these ideas, they tend to be more healthy mentally. 

Also, it is possible that people can be creative in their interpersonal relation- 
ships, in child rearing, and in relationships with key figures in their lives. Here again 
we feel that if they are able to be creative in that way in this quickly changing society 
of ours, they do tend to be more healthy mentally and to help others in this regard. 

It is also our impression that in communities there is perhaps a community level 
of productivity or creativity which could be measured and found to be related to measures 
of community mental health. We are now interested in trying to develop ways of research 
ing in preventive mental health programs in communities and in other social systems . 

I don’t think that I need to say any more about this. Some of us there are interested in 
creativity, and I will be glad to circulate among our people any ideas or projects, etc., 
that any of you have. 

C.- “You tell them that I would like to see a project set up on the ability of a person 
to produce novel, social responses to changing situations. 

C^-It is an area of creativity that is as yet untapped. It is an open-ended situ- 
ation where a person might write in responses, and these could be scored qualitatively 
or for productivity. . . ! 

(3- -I gather that the behavioral sciences part of the Ford Foundation was at one 
time, and still is, interested in this problem of creativity, and that there is some in- 
terest in your organization; we hope that there is some interest in the National Science 
Foundation. Is it possible that these organizations will ever get together and look at 
the field as a whole, as I think we are trying to do here? I think that one of our prob- 
lems in this general area is the likelihood that we can get support for only very short 
periods of time, and then the support runs out. I think that as we begin to unfold some 
of the problems here, it looks as though there will need to be a long period of develop- 
ment before there really would be a great deal of payoff. I would hope that some of the 
organizations might be able to get together and think in terms of not a year or 2 years, 



but even a 10-year program. 

SI- -We do make grants for projects of several years' duration, but have not yet 
joined with other foundations in planning a total look at the creativity problem. 

C--I’d like to peddle an idea to anyone who will listen to me. I tried to persuade 
one foundation, and there is a possibility that they might do it some day, but I don't 
think that it is very immediate. I think that a very good system in this area where we 
see there are a lot of pilot studies and a lot of half-done things, etc . , is for some 
foundation or group to announce 10 $10, 000 grants (or something arbitrary like that) 
for proposals that are in by such-and-such a date - -long enough ahead so that you have 
time to write a good proposal . 

When a foundation sets up a program of small grants, your bet is, P suppose, 
that 1 out of 10 is any good. Most foundation people feel that they are batting pretty 
high if they get 1 out of 3 that is any good — these are the figures that are always quoted 
in foundation circles. But then they would be prepared to back up the 1 in 3 or the 1 in 
10 which at the end of 2 years really showed promise, I think that this is the way to do 
it, because at the present time people will tend to go their own separate ways at differ- 
ent times and different places. I have no idea whether any foundation will do this. 

C- -I have an alternative approach, in which 1 or more foundations would assem- 
ble a group such as this, who have made some beginnings of work in an area, to state 
the problem. Then give them enough funds so they can work together for a year, that 
is, coming together for periods of time and just talking, thinking out the problem, setting 
up the parameters, the scope of the research, and then each individual researcher con- 
tinues to get support for an aspect of the problem which is later dovetailed into a coopera- 
tive study. Around the table here we have had different approaches, but they are not 
randomly enough selected that they cut into the pie in a different way, but each is going 
to add an important facet to the problem. If we were all working together with all the 
same instruments, all the same kinds of things, we'd really crack this problem. I’m 
afraid that even when we are all through some of us will still continue working on the 
loopholes, etc. , and much of our energies may be for naught. 

C--I think this is a good point. McClelland grabbed me eagerly after my speech 
and said, "Do you have the TAT’s for the creative and the non -creative chemists and 
mathematicians? We’d like to work out the need achievement scores on them. " I think 
the point is that if this is part of a plan, you can be sure of getting this kind of evidence, 
whereas if each one goes his own way, people like McClelland will never get the data 
they want. 

' 2 " Tm in favor of exchange of information, but I'm not in favor of planning. 

You can plan only to the extent of agreeing to use some of the same measures in your 
batteries. 

C--I wouldn’t think of control; I would think of enough communication so that if 
you are interested in need achievement for a particular kind of group and it is possible 
for us to get it quite easily, then there is this kind of interchange. 
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C— This cpmes from meetings like this, which I think are good if they could 
happen every year or every other year. 

C--I think another critical problem is the bewildering array of test instruments 
that has been announced here thus far. I don't know about you folks, but I know the 
chagrin you experience when you have committed yourself on some instruments in a 
testing area and you come across a study which describes a variable that you are so 
sorry that you did not put in. While you might be in disagreement with some of the 
hypotheses, the instruments themselves might be attractive. I was hoping that some 
amalgamation could be made whereby each person might list the types of instruments 
employed by him as well as what they correlate with. I don't think that it would be 
much of a job for each of us, and somehow or another this information could be made 
available to us periodically. In that information I think that there would be a lot of other 
material which would indicate the nature of the study generally by virtue of the simple 
write-up. I dare say that we have covered about 300 variables. The total picture is 
bewildering. I’m not suggesting a total amalgamation necessarily, but I wonder if we 
could do something in a more limited way. I wonder if we could produce such skeletal 
dittoed reports — each one of us. I don’t think there would be too much duplication. 

For example, I’m really interested in these personality variables that have been men- 
tioned. 



APTITUDES COMMITTEE REPORT 

Reported by 
Frank Barron 



S--The Aptitudes Test Committee set four objectives for itself. The first ob- 
jective was to draw up a list of areas of measurement in which new tests of creativity 
are needed. The second objective was to make specific recommendations for the social 
organization of research on creativity. A third objective was to state unresolved issues 
in such research. A fourth was to describe new areas in which research on creativity 
is needed. The report in outline form is presented below. 



Needs for New Tests 

1. Tests calling for a high level of integrative ability, especially ones which 
would require the integration of diverse stimuli, with something like the Rorschach W 
response, perhaps, as a prototype of the sort of thing required. However, such a test 
should demand a much higher level of integrative ability than does the Rorschach, and 
it should avoid the psychometric shortcomings of the Rorschach. 

2. A test for span of ideas or ideational power; we have in mind the number of 
ideas a person can hold in his head and manipulate effectively. 

3. Tests requiring a longer gestation period, calling for both continuous and 
interrupted work on a problem which would require a long time to solve. 

4. A high-level vocabulary test with a much higher ceiling than any we now have. 

5. A test for specialized abilities and interests -- the sort of thing that was 
mentioned in Wilson’s work in the Portland Public School System. The basic idea is 

that almost everybody has some sphere of functioning in which he can manifest creativity, 
and in our test battery we must include media which will offer almost every subject some 
opportunity to express the creativity which he has . 

6. Tests of risk estimation and tendency to take risks. 

7. Tests of decision-making on the basis of incomplete information, involving 
the construction of tests where die respondent has a freedom of choice as to how m . :h 
information he wants before coming to a decision, so that in a standard problem situ- 
ation he would rank -order people in terms of how much information they require be- 
fore they feel they have enough to make a decision. 

8. Non-verbal originality tests. We did not specify which tests we had in mind, 
but I suppose something like a mosiac construction test would serve as an example. But 
in any case, tests which do not require the manipulation of veibal symbols are needed. 



9. Tests in other sense modalities, e.g., touch. It is desirable to give the 
subject freedom to choose the sense modality in which he will work in order to solve 
a problem. If you can construct a problem which can be solved by the manipulation 
of verbal symbols, by geometric analysis, by spatial visualization, or by any number 
of other ways, and if the subject is free to choose his own method, this very choice 
might be an important variable. Here we had in mind the Einstein testimony as to how 
his ideas came to him. It is clear that some people can see those things mostly spatial 
ly, others mostly in verbal terms, and so on. 

10. An analogies test to measure preference for complexity in analogies. The 
idea here is to construct an analogies test in which all of the alternatives would be cor- 
rect. That is, if a person chooses one of four, all of the alternatives will fit. but they 
will fit at varying levels of complexity. 



Recommended Social Organization of Research on Creativity 

1. We thought first of ail that we should discuss ways of facilitating communi- 
cation among persons working on the same problems. 

a. There should be acceptance of individual responsibility to send reports 
of one's work quickly to other workers, and at various stages of the work — that is, 
not only in the final stage of the work, but at times when one is working on hypotheses, 
going along choosing samples, and so on. 

b. Compilation of a mailing list of interested workers. 

c. Taking advantage of professional meetings, or actually arranging such 
gatherings for the purpose of exchanging ideas and research experiences, such as this 
conference has been. 

d. Establishment of consultation services for technical problems in test 
development, having in mind here that there are among us certain specialized abilities, 
i. e. , persons who are better at some aspects of problems than at others. We could 
arrange it so that we could call upon one another for consultation, with perhaps some 
formalization of this in terms of recompense for the consultants. In all of this, there 
would undoubtedly be some financing lequired to carry out the social organization. 

2. We thought we should discuss cooperation among workers in using one anoth- 
er's tests and samples. 



a. There should be an establishment of criteria as to when one can properly 
ask a fellow worker to use one's test. 



b. There should be a statement of policies on test construction. It is clear 
thdt a test should be at a certain level o* ilopment and have certain characteristics 
already marking it as a sound device *,e one can ask another person to use it. In 
other words, there has to be a good reason for asking for cooperation. 
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c. There should be an exchange of information concerning hypotheses. If 
we ever arrive at any coherent theoretical statement which generates predictions, not 
all of which are likely to be tested by any one worker, it would be advantageous to parcel 
out among ourselves some of the work of hypothesis -testing. Thus, integrated theory - 
testing would be possible. It may all be a Utopian idea, but we had in mind the vision 
which was earlier evoked of the surrender of individual narcissism to a marked degree 
in the interest of social gain. This is a small step in that direction. 

3. There should be some recommendations made by later committees concerning 
how one decides as to publication credits on joint research . We felt it would be good if 
recommendations were made by some central committee rather than leaving it to individ- 
ual discussion. You could still have such individual discussion, of course, but in any 
case it weald probably help matters if there were some general set of recommendations 
as to how one decides such problems . 



Unresolved Issues 

(The committee attempted simply to name the issues without discussing them. 

In most instances you can probably see clearly what the implications are.) 

1 . Predictive efficiency of factorially simple vs . factorially complex tests in 
relation to the criterion problem. 

2. The predictability of simple vs. complex criteria. 

3. The apportionment of predictive variance to aptitude vs. non -aptitude tests 

at varying levels of ability. We have in mind here, for example, thac at a very high 
level of ‘ity the person who is going to be creative will be so regardless (practically) 

of moti on factors, while perhaps at lower levels of ability motivational factors are 

more important. I’m not suggesting that this is a good example, but what we had in 
mind was that at different levels of ability it might well be that motivational, tempera- 
mental and other factors acquire different importance. 

4. The appropriateness of tests to different developmental levels, with reference 
both to the research which was projected earlier in the meetings and also to some of the 
problems which Wilson will be runaing into concerning whether the tests he used as a 
factor for one age are appropriate for use at another age. 

5. The suitability of multiple regression vs. discriminant function techniques 
in certain problems . 

6. The ievei of reliability in tests needed for validation studies. 

7 . Statistical significance of the yield in terms of significant correlates from a 
large set of predictors. Some of us think we know the answer to that, but it’s a question 
that continually comes up. 
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New Areas in Which Research on Creativity is Needed 

1. Creative test performance under trance conditions, under the influence of 
drugs, etc. 

2. The strength and the fate of originality in women. 

3. Variation of scoring of present tests to make them more criterion -relevant. 
This is suggested by the new way of scoring Controlled Associations which one of you 
did, and there are probably other kinds of tests which might represent behavior of a 
sort which is not at present being scored and which in some cases might be more cri- 
terion-relevant for certain criteria. 

4. The beginnings of creativity in children, with an attempt to get at things like 
the whole matter of when scientific thinking begins, and how long animistic modes per- 
sist, and what the overlap between them means, etc. 

5. Creativity at an unlikely age. This includes both precocity and the persist- 
ence of creativity to an advanced age. 

6. Periodicity of creative test performance and of creativity in general, since 
we think there is some sort of ebb and flow. 

7 . The apparent inexplicable associations of creativity. 

8. Group activity and group ability as structural determinants of group creativity. 
Here we have in mind such matters as these: (a) if you mix up abilities in groups, ypu 
may get a diversify which results in greater group creativity; (b) authoritarian as opposed 
to democratic group structure may influence group creativity. 

9. The educability of creativity. The question here is, can people be made more 
creative by being taught - -can one teach creative thinking? 

10. Identification of false negatives in tests where false positives are at a mini- 
mum. There may be special ways in which one could approach this. 

11. Optimum level of creativity in an individual. "What price creativity? ” is the 
question here . Can there be too much creative activity, can it result in some sort of 
diffusion of other activity and loss of effectiveness in other important areas of personal 
functioning? What is too much and what is too little in creative activity? Is "perfection 

. of die life or of the work, " in Yeats’ phrase, a realistic statement of the choice which 
a highly creative person must make? 

Since some of our members must leave early, our group decided to work longer, 
so that we consider this a final report instead of a progress report. 

Q--Some of these not only involve aptitude measurements, but they involve mo- 
tivational studies, too, wouldn’t you say? 




S--Yes, we decided not to restrain ourselves . 

C_ — I think that you finished on a note here that we've neglected in the present 
push of creativity, with an emphasis on output, a high fluency level, hoping that with 
a high fluency level more responses will be creative, will be clever, etc. And of course, 
the emphasis in any science on productivity, the greater number of articles, etc. , wheth- 
er the data are faked or not. . . This morning Ghiselin and I were talking about the modu- 
lation of energy flow — that is to say, many creative acts emerge from the modulation 
of this thing, and it's very strongly apparent in playwriters and those who write scripts. 

I think Ghiselin could support this point of view in his studies of creative writing, I 
have in mind the terrific amount of tension that can be created in a movie or a play by. 
playing down of emotionality; this is built in deliberately by the playwright, and what 
he’s doing is being very creative in producing a highly interesting play by modulating 
some sort of tension levels. And then again it ties in with what Taylor was saying: 
the subject who gives you a few responses, which are creative responses for the most 
part. Here again is the modulation, you see, and I think that it’s been neglected thus 
far in the conference. I think our whole emphasis has been on output generally. 

C—You could even stretch that by saying we might test the ability not to produce 
at the tactful moment. 

S--That thought is similar to an idea that Erikson has put forward as being very 
important in persons who later achieve a high level of creative work. He calls it the 
psycho -social moratorium, and his main point is that such people refuse to react or 
express themselves. at certain times. They may hibernate for five or six years in some 
cases. They remain uncommitted until the proper time for full commitment has arrived. 

* C — This sound3 more tantamount to latency than die thing I had in mind. It’s a 

negativism, almost. 

. £- -No, that’s very perfectly illustrated in a lot of talk among writers that they 

must not talk about their story or their poem or whatever, because one reason that 
seems obvious would be that preliminary or too early definition means a tendency to- 
ward fixation and the rest of the background stir. But also it means that there is an ex- 
penditure of motivation that is frittered away instead of expended in the crucial develop - 
ment of the absolutely perfect insight. 

C— We call this the Zeigarnik effect in psychology. The experiments show that 
if the individual is interrupted on a task and talks it out, he is less likely to go back and 
complete it than one who hasn’t been interrupted. There is a good deal of evidence of 
this in graduate dissertations. The student who goes around and talks about his disserta- 
tion doesn’t seem to have much energy left to write it. 

C- -There is a lot of evidence that the person under analysis who goes out and 
blows off steam to some third person is subverting the progress of his analysis thereby. 

C Then, this conference has just set back our work two weeks . 
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C--But you will control your incubation period a little longer. 

C— I had one small footnote. "When you spoke of the risk-taking thing, you were 
going to distinguish between betting and cases where it really depends upon your own 
efforts . I think both should be done . You could set up a test in which you choose which 
of several things you could do . You could make it clear somehow that there are dif- 
ferent probabilities of success, if you chose to do that. But it depends on the subject 
as distinct from the pure betting situation which is something different. 

S--I would think that on one part of such a test the examinee should have the 
option of wagering >r not wagering on situations of varying probability. And on another 
part he would have to wager on every event, but would have the option as to what propor- 
tion of his capital to wager . 

C--The crucial dimension is whether it is his activity that is relevant. For ex- 
ample, we have done research which shows that the high need -achievement person will 
not wager at all. He just won’t bet any of his capital. 

S--He may not wager on events over which he has no control, you mean. But 
this does not imply that the events would be ones over which he would have no control. 
You can set it up on some such model as that of a poker game, the outcome of which is 
not really a chance event at all. It is a problem in estimation on the basis of social 
information. 

C--Well, all I'm saying is that the test should be divided into two parts, one in 
which the person clearly has control, not complete control, but with a very considerable 
measure of personal skill entering in, and another in which it is beyond his control. 

- S--Well, I guess the old level -of-aspiration tests involve something of this sort. 
If someone should make book against the subjects, the level -of- aspiration experiment 
would become a risk -estimation situation. - - 

Q - -Would you care to predict a mean for the level -of-aspiration test for our sub- 
jects? Which would have a higher level of aspiration, the highs, or the low creative 
group? We give them the digit symbol test. They take the first trial, and then they are 
given ’’fake” dafa where they are told that they have exceeded 60% of a hypothetical popu- 
lation that has taken this test. How much will they do the next time? 

C_--Well, this is an example, I think, of where we have economized to a bad 
advantage. There would be a crucial difference if it were a situation like the original 
German task, where they actually had to choose and do one of several measures - - 
rather than merely state a verbal level of aspiration. 

C--But in this situation what do you think would occur? The interesting thing is 
that the low creatives do have a higher level of aspiration on this, but the highs improve 
more as they go along. 

C--You see, there is no real commitment involved in just a verbal statement. 



But it is much different if you give them their choice of which of these tasks they are 
actually going to do, and it is clear that they will either succeed or fail. 

C--Yes, but this is consistent with data we have where we ask how much they 
hope to earn in ten years and how much they expect to earn in ten years, which again 
doesn’t have the commitment factor that you are after. But, it is still consistent with 
that since the lows overestimate, in having a higher discrepancy between ’’hope” and 
’’expect” than the highs, which was consistent with your need -achievement data, where 
they are much more cautious and will not stick their necks out. 

C— You see, with the present criterion being mainly productivity, the high cre- 
ative groups you are going to get are probably those people who are choosing projects 
and working on those, and aspiring projects within the realm of immediate skills and 
realizations, whereas the low creative group might be going off on long-range kinds of 
studies that hang them up, so they don’t tend to produce as regularly. On the other hand, 
their aspirations are going to be higher . 

C— In relation tp an earlier point, ETS has data for a test which separates risks 
that depend upon your own activity and risks that depend on other people or chance. 

These are two quite distinct measures. This is a verbal test somewhat like level -of - 
aspiration tests. It can be referred to as a risk -level test. 



PERSONALITY COMMITTEE REPORT 



Reported by 
Robert H. Knapp 



S--In the first meeting of our committee we decided that there were three prob- 
lems that we should think about. First, what are the psychological characteristics of 
scientists in general? What is the psychodynaihic motivational theory of the scientists? 
Secondly, what are the distinguishing motives that separate creative scientists from un- 
creative scientists? Thirdly, what can we say about situational factors as they will af- 
fect the scientist in one type of environment as opposed to another? 

I think we became impressed with the fact that the psychological attributes of the 
scientist as seen generally and the attributes of creative people as described by Barron 
are very dissimilar. We may conceive of this relationship if I draw one of Eular's dia- 
grams showing one complex here for scientists and another complex over here that ap- 
parently represents die psychodynamics of creativity. The creative scientists would be 
in this small overlapping region. 

We spent most of our limited time talking about the general complex of attributes 
characterizing the scientists, i. e. , point one above. What we did was to sit down and 
make a list of the motivational attributes of scientists in general as they have been re- 
ported in a number of separate studies. That’s about as far as we got. We believe 
there is pretty good congruence in certain of the attributes as they emerge from very 
different and independent studies. For example, McClelland has selected the following 
items from the Strong Vocational Inventory that particularly characterize scientists: 

1 . A dislike of interpersonal relations. 

2. A fondness for the mental manipulations of things - -not of people. 

3. A liking for method and exactness. 

4 . A liking for autonomy . 

5. A dislike of sterile or monotonous rouble. 

6. A dislike of intro versive and affect -associated preoccupations, for example, 

poetry. 

7 . A certain interest in the problem of taking the wager, of gambling. (It is 
this thing we have been talking about all along, this pleasure in pitting one's self against 
an uncertain situation in which one's own effort can be a factor in the solution. It is 
comparable to the entrepreneurial wager - -in other words, the liking for ventures in- 
volving calculated risk. ) 

Then there are five psychodynamic postulates that came out of my data or could 
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be found also in the data of MacCurdy of Anne Roe . These are: 

8. Scientists tend to prefer the mechanism of repression and isolation in deal- 
ing with instinctual energy and affect . 

9. Scientists have a peculiar area of sensitivity in the domain of aggression; 
they have a distaste of interpersonal controversy in almost any form. 

10. They have a low Oedipal intensity. That is to say, they have a rather low 
tendency to rebel against the father figure and also a rather weak tendency to cathect 
the mother or the mother -surrogate as a love object in any identifiable form. 

11. They have strong perseveration and strong control of impulses. Roe, for 
example, finds a radical difference between the perseveration of scientists and the 
peculiar impulsiveness of artists in their work habits. 

12. Scientists show strong extratensive orientation toward the world of things 
and objects . 

We now turn to Saunders' study and we find again some qualities that separate 
his research scientists and engineers from those in other callings. These all appear 
consistent with those mentioned above. 

13. They are less talkative, sociopliilic. 

14. They are low on impulsiveness. 

15. They are low on gregariousness. 

16. They have a liking for abstract thinking. 

17 . They are not especially driven by status aspiration. 

18. They have a high tolerance of ambiguity. 

C--May I say parenthetically that this last is also evident in TAT protocols 
where scientists would not bring things to a conclusion; they could leave them open- 
ended, or they could give you five answers and ask you to take your choice. When I 
reported this to our own faculty, the scientists all said, "Isn't this nice? We trained 
our students to withhold judgment until the facts are in, and they do This is evidence 
that scientific training has done something for these men didactical 

C--But because a man gives you several alternatives anc jays to take your choice 
doens’t mean he is tolerating ambiguity; he may be withholding judgment. 

C— Can you withhold judgment without being able to tolerate ambiguity? Isn't 
that a necessity? 
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C~No, because you can say the facts are so, and I may refuse to go further 
until I see additional facts . 

C'-What were your test items covering this point? 

P They were such things as: 1) Are you annoyed by a person who is always 
doubting and not sure about things? 2) Would you rather bet on something that's an al- 
most sure thing or something that’s 50-50? 

C- -But a "no” answer to the first question that you mentioned doesn't mean that 
he is tolerating the guy, you see. He may be dismissing him entirely. I just mention 
this because tolerance for ambiguity doesn't seem to enter into the total picture d can 
see thus far. Withholding of judgment; suspension of judgment; depersonalization qi 
an irritable thing, person, or element — these I can see, but this One thing is hard for 
me to absorb into the total picture you have indicated. 

i r 

C— I see what you mean and I think you may be right, but how are you going to 
describe the fact that one can view human situations, that seem to have inherent tensions 
in them and then somehow not feel compelled to bring it to a fullness of conclusion. I 
say one can tolerate a lack of closure . 

C-It might also be phrased as resistence to premature closure. 

CJ- -Keats in one of his letters has a statement that might be helpful here . He 
speaks of the ability of the creative person to tolerate a high degree of uncertainty, be 
happy, or to be at ease in the presence of very complicated uncertainty, and not to re- 
quire resolution. It suggests Barron's idea of the huge mass of disorder that could be 
positively preferred. You could say he is at ease with complexity. 

C--You see, we are talking here about attributes that seem to characterize 
creative people, and now we are talking really about the character structure of scien- 
tists. This ..may be one thing both have in common. 

C_ I think that is the essential thing that the creative scientist and the creative 
artist have in common. I think they differ in many other respects -- for example, the 
suppression -expression dimension. But it is precisely on the matter of how much 
wealth of observation they will permit into this system, their perceptual system, before 
attempting to impose order upon it that they are alike, as well as in their resistance 
to arriving at a structure which cannot include everything. The highest level of art and 
creative science is characterized by this quality. 

t 

C--Is this a variable that differentiates the creative scientist from other scien- 
tists, or do you think that it is one that separates scientists from other people? 

C— This is an empirical finding from Saunders’ research; he found that in toler- 
ance for ambiguity his research scientists had a higher standing than the others. I've 
forgotten how it came out on the second group. The applied scientists, as opposed to 
the research scientists, were lower yet. 
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C- -Thef e is one other point of evidence on ‘this artist -scientist thing --this com- 
plexity-simplicity measure - -which correlated with ratings of originality; research work 
was actually derived by giving a group of some 80 artists in cities ail over the country 
a set of figures which were derived by item w analysis comparing artists with other people 
in general. And it so happened that it reproduced the complexity -simplicity factor which 
emerged from a factor analysis of the full Welch Figure Preference test. Now, what 
happens is that scientists who rated high on Originality prefer the same kinds of line 
drawings or designs or phenomenal fields in general as artists. Those who rate low in 
originality tend to prefer the simple figures . 

C--Have you tried this with populations of scientists outside of Berkeley? I'm 
troubled by the ideas in an academic environment among students where the going in 
for these kinds of activities, Bohemian activities, makes demands in this area. But if 
you got out of the academic, I wonder if it would still hold? 

• j ' * 

S- -Maybe we ought to return to Saunders' variables, bearing in mind that we are 
describing scientists, hot necessarily creative people. We have then: 

19. Scientists in general were higher in self-criticism than others. 

20. Scientists were higher on the masculinity -femininity dimens ion than the aver- 
age, while research scientists were higher than the applied, and the better scientists 
were higher than the less distinguished. 

We have four more attributes of scientists that Stein has given us . He finds his 
scientists are characterized by: 

/ - ' * • ' i * i * . / * ' ' ’ 

21. The depersonalization of self-identification. 

22. Delight in the mastery of ideas or pleasure in ideational mastery. 

< * * s * \ . * • * 

23. Controlled affect. 

24. The special devotion to Secondary thought processes to the exclusion of the 
primary thought processes, at least within the context of science. 

This concludes our preliminary listing of the attributes of scientists. 

C£--What do you mean by "primary" and "secondary?" 

S-- Freud argued that the distinction is this - -the primary thought processes are 
essentially in the service of instinctual energy and, broadly speaking, operate according 
to the needs and the logic of the pleasure principle. The secondary thought processes are 
those that are devoted to the examination and faithful grasping of outer reality and can be 
thought of as related to the ego’s function in obtaining a faithful picture of the environment. 

. * 7 * g l , 

C--I had one minor point, that under appropriate conditions he will indulge in pri- 
mary process activity. The difficulty in experimenting with scientists is to create the 
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appropriate atmosphere for getting at their creative thought process . That is, when 
the scientist talks to us, he still will talk to us in terms of the, rational processes he 
uses. He has to communicate it, and the communication has to be a rational process, 
so he will indulge in that. However, if we create the proper atmosphere, we may get 
closer to what really goes on in his head. The kind of thing I'm referring to here is 
illustrated by one of our scientists who was in good rapport with us . He said that when 
he is working on his material, he sees atoms and molecules moving around like friends 
of his, and he works with them as friends. He is very animistic in his thought about how 
they shift positions, and he knows that ’’so-and-so is a good friend of so-and-so’s and 
therefore they will mix. " But it was only when we got him almost drunk that he could 
talk to us in this fashion. But if we start off by asking him, ’’Well, what do you do?” 
then he talks rationally. Then he writes out a formula and says this carbon ring, as 
demonstrated, has to go. Later he says, "Well, I don’t really think that way when I 
work. ” He says, ”1 have friends over here, and I know they are going to come up at the 
right time. ” He is all at one with his work. That’s what I mean by appropriate condi- 
tions and it raises all kinds of questions for us about appropriate instructions in our 
test administration. 

C--As a matter of fact, Anne Roe reported the same thing -- she couldn’t make 
anything of it. You remember, she was very much struck by the vividness of imagery. 

It was different for different scientists, and she talked for a iong time about the need 
for a good test of imagery. 

Cl --Well, I think that the distinction that Reichenbach makes between the context 
of discovery and the context of clarification is crucial to this, so that the scientific way 
is to work in the context of verification. When we talk about the development of scientific 
talent in children, a reasonable hypothesis is that future creative scientists adopt scien- 
tific canons earlier and retain their animistic modes longer. These things exist simul- 
taneously at high strength in the person who is able both to discover and to verify. 

C--My feeling, even further, is that what actually leads these people to take this 
risk is this animistic thinking. In other words, it isn’t a rational process at all. !t 
gives them confidence because they have gone through a lot of pre -logical unconscious 
thinking already. 

C--But they have also had the switching mechanism that was alluded to here. 

That is, they have to have both simultaneously, some way they switch in and out of 
animistic and rational thought. 

C--In Freud's essay on "Formulations Concerning Two Principles of Mental Func- 
tioning" (I believe it's called), where his theory of thought is expanded, he talks in terms 
of cathectic dynamics. The hypermotility of cathexis is what characterizes the primary- 
process most. When you get into the secondary process, the thing that occurs is that 
this motility of cathexis is done away with --that is, you don’t get symbolization, con- 
densation, and displacement, etc: instead you get what he calls "experimental acting 
through thought. " His idea is that thought arises as a result of acting in imagination, 
which is a form of experiment. Then he goes on to say that the essence of science is 
the complete adoption of the reality principle. 
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C- -We have one way of getting at this in a sentence completion test where we have 

an item that reads., "having an idea is like ? " and the subject has to complete 

it. We differentiate our highs and lows in creativity in terms of metaphors . The high 
subjects will often say, "Having an idea is like a fresh breeze coming into the room. " 

Our low subjects will say, "Having an idea is like looking at a new book" --a very con- 
crete thing. 

C--Does anybody say, "Having an idea is disturbing to me, destructive to me"? 

C--Yes, people who are of low creativity. 

C--I suspect that these low creativity people who are coming out with these re- 
sponses are suppressing their creativity because they are too much aware of the censor- 
ship or the criticism they may face. If you give those low people a manifest anxiety 
scale, I bet they will come up high. 

-Actually there are no differences. As a matter of fact, we have given them 
the Freeman anxiety scale, and the high people tend to be somewhat more anxious . 

C--One key to the puzzle which I haven't heard anyone answer is why does the 
scientist develop so much anxiety over interpersonal situations so early? What happens, 
what causes this anxiety? Possibly it is connected with the mother, but we don't really 
know why this anxiety really rises in him as opposed to others. 

C^--I think one of the interesting things in relation to this problem is Knapp's 
suggestion that maybe we are going to produce fewer scientists in the future because 
our child -rearing practices tend to produce less repression and isolation. We may 
get a different kind of scientist in the future if this does occur . 

C — I wonder if you couldn't turn that around and say that the difficulty is that 
a man who sees as a scientist is isolated from so many people and is so different from 
other people that he feels his difference and is rebuffed and learns to withhold himself. 

C--But it comes early in life; you can almost tell the really hot scientist by the 
time he is three years old. 

C - -There is one item that helps us in this direction, if we can accept the data 
which seem to be relatively consistent, that the greater scientists tend to be the first 
or last children. Let us consider the first child. There are two items that I haven't 
seen yet. The first is how long after the parents were married he was born, and the 
other is the age of the parents at the time of his birth, which would be critical data. If 
you accept the cliche' that the first child is the experimental child, the other two items 
are critical data here. Then we get some leg on the problem of what's making one of 
the sources of the anxiety; that the parents have as yet not adjusted completely to the 
child, and it's going to be a much more ambiguous situation. Therefore, he will have 
to seek security internally and with intro jected material. This is one leg on the prob- 
lem as to why the scientist is more exposed to this kind of anxiety than others . 
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C- -Erickson, and also Kluckhohn in his book dealing with his view of western 
culture, say that when the child r< iches the adolescent age, the mother lets him go off 
on late -hour parties and drive off with members of the other sex with impunity, knowing 
within herself, at least on an unconscious basis, that she has succeeded in starving the 
child sexually during the earlier years . I ran across this in a course I had to take . It 
may be that this sexual starvation he talks about is the first push toward the deperson- 
alization which we have talked about. 

C- -You have a cultural problem here. It is the peculiar prolificness of certain 
varieties of Protestantism in the production of scientists. ’’What do they do by way of 
rearing their children that makes this particular form of adjustment somehow a common 
thing among them?” As Merton had shown abundantly, Calvinists, particularly in the 
first stage of secularization, turn out an extraordinarily large number of scientists. 

Now, there must be something in their pattern of childrearing of ideology which operates 
here. You don’t get it among Catholics. 

C* -I think there is another aspect that we have overlooked. Fromm hasn’t come 
up in our discussion too much, and he talks about the terrific marketing organization 
which prevails. You’ve got to prepare for the market. Your sales -ability index is im- 
portant, and early in the game you’ve got to be taught those processes which will make 
you of value on the market; this may tend toward the kind of personality structure we 
are considering here. 

C^-But isn’t part of this boiled down to the kind of scientist we are talking about 
here? I think that we’ve all overlooked the differences among the scientists and within 
the scientists. For example, let’s take this problem of childrearing and attitude towards 
the mother and father. If our subjects say that they were closest to their mother in 
childhood and later started to identify with the father, we think they are going to be 
highly creative biochemists . If they went into physical chemistry we might predict they 
would be low. So far it has been coming out that way. With the physical chemists, on 
the other hand, if in early childhood they tell us that they identified with the father all 
the way through, then they will be highly creative physical chemists . I discussed this 
theory with Sears. He came up independently with this notion, suggesting to me that I 
study dependency relationships and make predictions for the different occupations . There 
seems to me to be something very important in this area. Fo: example, in one study of 
a theoretical physicist and a non -theoretical physicist, this thing breaks out again where 
the theoretical physicist has much more depersonalized identification and more positive 
attitude toward the father figure than the practical physicist who still has embodied in 
him more positively cathected attitudes toward the mother . This was by Carlo Merra 
at Chicago. As soon as we get a large enough population of biochemists to contrast with 
physical chemists, we will really be able to test this more thoroughly. So far, on a 
clinical basis, we can really separate them in this regard. If you take the biologist, he 
is going to be different from the physicists to start with, and the real difference here, 
among other things, is how much of the cathexis of the mother stays with him. 

C^--I think you are right, and it may be not so much the amount of cathexis, but 
how much he can tolerate awareness of it. 
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C- -These particular questions, it seems to me, are not as relevant to the identifi- 
cation as they are to, perhaps, the understanding and even the eventual development of 
new theories . 

C- -Surely, they lead to the development of biographical questionnaires. 

C--In addition, though I dislike saying it, I think that there is involved here al- 
most a mechanistic model of the human organism. We start off with the notion that the 
organism does get programmed very early in life, just as the mathematician programs 
the high speed calculator. The scientist, especially, gets programmed through these 
experiences. Now, of course, we are not denying that later socialization can also have 
an effect, and this is where many of us differ from the over -orthodox Freudians. But 
we have to study the programming if we are going to identify talent early. If there are 
no specific changes in later socializations to change the organism, to change the pro- 
gramming, these will prove still more stable effects . 

C— This, then, gives us the basis for symptomatology that we could employ in 
biographical questionnaires or a great variety of other techniques. But, much more 
important, you get into a more basic theoretical level, and this may lead to tremendous 
consequences. For example, how would you produce more scientists under a particular 
set of cultural conditions? 

C--Well, I think in this matter that the most important variable is not history 
of childhood or childrearing practices, but that it is intelligence simply. At the highest 
level of intelligence one of the characteristics of intelligence is that impulses are inhib- 
ited and judgment is suspended, and one goes away from unconscious and spontaneous 
participation in things toward something like ideation of conscious knowledge . 

C--Yes, but can’t you envision two people equally intelligent, one raised in 
Sparta and the other in Athens, and as far as I*m concerned, you are going to get your 
social scientists out of Athens and get your physical scientists out of Sparta. I can't 
envision the Spartans coming out with great debaters of social issues . 

C— I think that the correlation has been greatly exaggerated between being a 
scientist and having such traits. I think there may be some correlations in the neighbor- 
hood of .25 to .35, but no more. 

C--You mean this dislike for interpersonal relations, for example, is correlated 
only . 25? I think it is of the order of .75. 

C--The kind of data we have would seem to indicate that these do come up rather 
beautifully, especially on physicists . 

C-- What's the nature of the correlation? Do you take scientists and take persons 
of equal measure of intelligence, and then compare the scientists with the non-scientists 
and discover what the correlation is of these traits? I am arguing it is not a function of 
that but of high level of intelligence, and that, as I see it, the scientific worker is a fore- 
fronter, the most advanced point in evolution, in a sense. The thing that is the most 
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characteristic of organic evolution is the development of the central nervous system, 
intelligence, and of the ability to delay, so that one could well argue that these things 
we have discovered in scientists are really a demonstration that they are a higher level 
of intelligence than the generality. 



C— I*d reverse it and say that the high level of intelligence is what is necessary 
to go through the courses, schooling, etc. , necessary to get into science. If there 
were outlets for people to get into it without going through our graduate programs, I 
think that this level of intelligence would change considerably. 



C--You also have trouble with history. Are you seriously prepared to argue 
that the Athenians were really smarter than the others and that they got less smart, 
something happened to the genes. . . 



C--The basic problem is that we do get cycles, and they are not easy to explain 
by variations of intelligence. 



C--You have to take social heredity into account in explaining intelligence, too. 
Why do you say that cycles can’t be explained by variations in intelligence? 

C--I won't argue this point of view seriously, but my reaction to this description 
of a scientist explained in terms of childhood history is to say that f suspect that the 
main thing that produces this picture is intelligence. I’m not at all sure that this is 
unique to scientists at all. 



C— I’ll buy this, that if you get a certain pattern of child relationship — call it 
"X" --and this is somehow responsible for producing scientists, it will only produce 
scientists if they are of a certain level of intelligence, a minimum level of hi gh intel- 
ligence. If you get the same childhood experience for people of lower intelligence, 
you get something else. So in this sense, it is a crucial variable in the development 
of a scientist. 



C — And at a later stage, in terms of our research, we are able to show that if 
we equate for opportunity - -for example, an I.Q. of over 125 - -intelligence is not cor- 
related with creativity. 

C--I wonder if I could make a little comment about this syndrome, as to how far 
it does apply to scientists as a whole. I think it does not apply very well to theoretical 
physicists or mathematicians. Frankly, I think they are a different kind of animal and 
have greater introtensive qualities . I think they have greater rapprochement with their 
pre-focal world, and so on, I think there are a lot of differences, including their Ror- 
schach patterns. And secondly, I think it doesn’t apply very well to biologists or people 
concerned with life processes, but I think it does fit in a pretty simple way the experi- 
mental physicists, chemists, geologists, engineers and people in this field. And then 
one other point: you have proposed here that scientists are nature's latest evolutionary 
advance in the character type. 

C--He represents, from the evolutionary point of view, a very high order of 
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intelligence, and the rest of his qualities follow this fact. 

C— Well, I’ve got two apprehensions there. One, you do not have a very good 
theory of the energy or of the dynamics of the problem. Where does the energy come 
from here? Intelligence is not normally an energy concept. Secondly, you don't have 
to look very far in here to see that, iri many respects, these scientists are really inade- 
quate socially. I can't conceive that their social blindness and social insensitivity rep- 
resent an evolutionary advancement or are products of intelligence, per se. 

-Something has to be given up in order to develop more highly in these crea- 
tive directions. 

£ I think you have a real point there on intelligence, because actually if you 
look over those characteristics we have listed, in many ways the scientist is the most 
mature individual, barring the social, because he can combine both the rational and the 
emotional. 

C/--Yes, in Anna Freud's Ego and Mechanisms of Defense she tries to point out 
specifically where intellectualization, while initially a defense mechanism, can become 
incorporated as a positive ego tool for dealing with the environment. 

C— There is an article by Henry Hard in the Psychiatric Journal in which he is 
talking about the general notion that creative people are neurotic. He says, "Who are 
the less actualizing people, all the people for five thousand years who used a quill for 
a fountain pen or the man who finally thought to develop the fountain pen? Who are the 
well people? Are the rest more sick, and he the most well?" 

S — We earlier presented above in our progress report some 24 attributes of 
scientists . We have not gone into the question of how superior are distinguished from 
inferior scientists, nor have we gone into the problem of psychodynamics of creativity 
as distinct from scientific things . We met again subsequent to our earlier meeting and 
sought to organize the 24 attributes into patterns or clusters. We have come up with 7 
packages and a few items left over in a miscellaneous category. I think the thing we 
would like to do now is to describe these 7 groupings, which seem to represent focal 
and agreed-upon characteristics of scientists in general. That is, this does not describe 
social scientists, probably not theoreticians, but research scientists, predominantly in 
the physical sciences, though some biologists are included. There are 7 cardinal traits 
that seem to crop up again and again from a* number of different studies and represent 

a consolidation of the initial 24. They are: 

• 

1 . A liking for manipulation of ideas, a liking for the toying of ideas, for its 
own sake . 

2* The general quality of impulse control, repression of affects, isolation, sup - 
pression . It’s all from the same package. This something crops up again and again, 
and I still don't think we understand what initiates it, how it gets established, but I do 
think it is extremely important in the personality structure of scientists. 
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A low level of interpersonal involvement in human relations . We were talk- 
ing today about engineers and scientists tending to join and affiliate themselves with 
professional bodies. The thing that shows up again and again is that they don't like to 
get into warm, close, human relations, they're not very good at camaraderie, they don't 
like to take leadership in group activities, they don't like competitive sports, body con- 
tact sports, etc. They like impersonal things like nature walks and sailing. This ap- 
plies to superior science students in high school, as MacCurdy shows. It applies to 
undergraduates, majors in science, that I've looked at. While this cluster may not be 

securely established for all levels or types of science, it is indicated by one or more 
studies in the field. 

4 * A d evotion to independence and autonomy . They don’t like to be bossed 
around; they are sensitive about it. They are not tractable and flexible in organizations. 
Sometimes they are non -conforming. They tend to be self-sufficient. 

5 * T° leran ce for comle xity and the capacity to defer closure . They don't have to 
make up their minds in a hurry about things for some reason. In this respect, at least 
m my own findings, they are contrasted rather sharply with the humanists who like to 
wrap dungs up m a bundle pretty fast. This again is consistent with Anne Roe’s findings 
that artists have to bring things to a quick conclusion or they never get them done. 

C- -These two things are the same - -delayed closure and the tolerated complexity 
As you mentioned, this closure that the artists or the humanists need is a fast closure 
w ereas the physical scientist can tolerate the lack of closure for quite a while. It re- 
minds me of Arnold’s distinction between what he calls organized creativity and inspired 
creativity. Now, leaving it open leads to organized creativity, whereas the mad dash 
01 the artist to splash his ideas on canvas is the end product of this inspiring type of 
creativity: there may be a real difference there, based upon the kind of act they perform. 

£- -I'm sure foe™ is * It all goes back to impulse control. The artist, in a sense, 
idces his impulse, and the scientist has to protect himself from it. 

C —There is a further and essential qualification that the artist hasn't any medi- 
um of formulation, any way of defining his configuration except the medium he uses. 

He can t paint his mind; it is partly kinesthetic, for example. 

C — So what if his painting is a step in reaching closure and does not represent 
closure itself? 



... C— Right, and he revises often, again and again, and for that very reason he dis 

likes successive thoughts. 

C--As far as he is concerned, he could throw the others away. 

C-On this point on complexity and deferring and delaying closure, do you mean 
this is a general quality, or does it have to do with thekind of intellectual problems he 

works with? My own feeling is that this is right within the work sphere, but is quite 
wrong outside that sphere. s 
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6. A liking for things and a preoccupation with the physical universe . This is 
also shown on practically all of the Rorschach findings that I know of: mine, Stein's, 
and Anne Roe's. In every case there is a high response to color and a low movement 
response . This is commonly interpreted as an outer responsiveness to the physical 
universe and an insensitivity to solicitation from subjective sources. 

C--You don't even have to take movement responses. We have been doing some 
work in which we take physicists vs. teachers, for example, and just the amount of 
human response at ail differs terrifically between them --no overlap. 

7 . Low projectivity, or a high commitment to secondary thought processes . 

This is partly my own notion, that we found that our scientists simply wouldn't play the 
game of taking the projective test. Give them the Blacky, and they wouldn't project on 
it. Also, it comes directly out of Stein's materials where he argues that the scientists 
are projectively unproductive and also from Freud's general theory that the scientist 
has to operate mote or less purely with the world of secondary thought processes . The 
artist, on the other hand, is more or less obliged to permit the intrusion of primary 
thought processes. At least I'll put it this way — it may be that in the inspirational level 
of scientific creative activity, he does exploit some of these primary mental thought 
processes. When it comes to criticism, evaluation, and formulation, the secondary 
thought processes have to be retained and employed in a very pure form. This is, in 
essence, a way of saying that he has to have a peculiar commitment to objectivity and 
cannot permit the intrusion of primary, impulse -dominated processes. 

C--Can you say that is rational vs. irrational thought processes? 

C--I don't think it’s quite that, but it's close enough. I think that there's a dif- 
ference in the kind of process: perhaps non -rational vs. rational. Of course, one 
argument is that the primary thought processes have their own law of rationality, and 
the secondary thought processes have a different system of rationality. 

C--Yes, it’s just as well organized. Could it be that the run-of- the-mill scientist 
is less flexible and thus more fully committed to the secondary thought processes than 
the creative scientist who at least occassionally does explore his primary thought proc- 
esses? 

C2— I'd like to suggest that as we closed the meeting this morning you talked about 
the terrific sense of honor that the scientist has towards his work, and so on. I think 
another category might be a combination of a strong super-ego coupled with a self-con- 
sistency. He wants to be consistent with this conscience of his; therefore, he has a 
strong sense of honor, and I wonder if that's another category. 

C^--Well, it comes right out of it. I've had the feeling that, whatever it's worth, 
the scientist in a peculiar sense has his personal honor involved in his fidelity to the 
canons of the secondary thought processes. For example, a scientist may look at re- 
ligion and find in this something that he would.like, in principle, to believe in. But he 
feels it utterly impossible to believe on faith and to surrender his rational position, 
because this would somehow be dishonorable. 
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C--You see, it stems out of something else. You said that they had little rem- 
nants of the Oedipal complex . Now the super -ego has been termed as the graveyard 
of the Oedipal complex, so evidently this Oedipal complex is laid dead in these people, 
and they come up with this good strong conscience. 

C - -They are not sin-fraught personalities; they are not sin-conscious person- 
alities; they somehow transcend the category of sin, and one way they do it, of course, 
is to get into this world of objectivity. They depersonalize the universe and themselves, 
and this is the way perhaps of escaping their super-egos. This is a problem that has 
to be understood better. I'm just guessing here. However, Ido think that the scientist, 
as a character type, is an extremely important one in our time and ought to be approached 
with the psychonalytic theory in mind. 

S--Now, finally, to mention the things we didn’t get to but that we know are im- 
portant. We know, from the analysis of situational factors discussed here two or three 
times, the effects of different social climates on creativity and upon the type of man 
who will be creative in different climates . One man will do well in an industrial situa - 
tion, another will require to be in an academic calling. There’s a whole host of prob- 
lems here that are deserving of examination. Then there is the problem of the analysis 
of the biological and cultural factors. We still don’t know much about the childhood of 
scientists; Anne Roe has done a little bit. 



GROUP DISCUSSION ON THE CRITERION OF SCIENTIFIC CREATIVITY 1 



C- -Certain jobs have what the sociologists call certain functional prerequisites 
or requirements. For example, in order to succeed as a doctor, you cannot sit on the 
beds of your patients and get emotionally involved with them. We should make a func- 
tional analysis of what it takes to be a good scientist - -not a critical incident sort of 
thing, but analytical. What characteristics would wreck science if a person had them? 
It seems to me that this is part of the criterion analysis; it is a rational type of approach. 
The farthest I've been able to get on this is that it is quite obvious that a scientist is 
somebody who must tell the truth. This is a functional prerequisite. If scientists didn’t 
tell the truth, somehow the very essence of science would be contradicted. Now, if this 
were true, and if we could refine what we meant by ’’telling the truth” and develop a 
criterion of what we mean, then we might get a predictor for this that we don't have at 
present in any of our batteries . 

-Could this be the role concept? For example, today Pelz indicated that there 
were two kinds of roles, the institutional role and the scientist role. A good many of 
our graduate students get into a student role and have tremendous difficulty ever getting 
out of it, especially after they have gone four years to the University. It seems to me 
that if we had this role concept developed, we might be able to do a number of things 
with it. 



C--How do you go about this, as far as the role of the scientist is concerned? 
You can't go about it by asking scientists, because they might not know; this role may 
be unconscious. 

C^--I think it cuts between role concept and critical incidents. In one of our 
science departments, the faculty constantly kept insisting that they were looking for 
original and creative individuals. Then a few hours later they told us the following 
incident: A student had the notion that he could do a dissertation without consulting 
with anyone on the faculty. There wasn't a faculty member who was especially compe- 
tent in this subject, and the student insisted that he could do a dissertation without this 
kind of aid. There was some suggestion as the faculty talked about the student that if 



*The entire conference group discussed the criterion problem in a general 
session on the evening of the second day of the conference. A sub-commit- 
tee was then discussed by the whole conference. The following report takes 
up these discussions in chronological order, with the minor exception that 
the committee made one early preliminary report which was briefly dis- 
cussed before the final report. In the paper that follows, these two reports 
and the discussion following them are condensed for simplicity of reading 
into one report and discussion period. Lindsey Harmon was primarily 
responsible for editing all discussion on the criterion problem. He re- 
ceived valuable assistance from Benjamin Bloom, Henry Dyer, and Joe 
McPherson. 



he ever got through, they would be very surprised. Here you can get a notion that they 
were seeking from the student a role with a certain kind of dependency relation to < Vm 
althougn objectively and at another moment, they were talking about independence, orig- 
inality, and things of this kind. In this particular case one incident helps to define one 
aspect of the role of the student . 

C-It also shows you have to analyze the faculty role, too, and the reason for 
their reaction - -the feeling that after all it is part of their job to see that people do 
not get through who do not know anything. 

C--The methodology we used in our assessment research was essentially that 
of setting up a hypothetical model from a study of the environment, and then predicting 
from psychological instruments the individuals who fulfill this model. You have to be 
very careful in differentiating between the verbal formalized statements that you get 
from the faculty and the informal, but really critical factors. For example, we were 
called in to assess students in a professional school; we tried to get at it in a variety 
of ways, but couldn’t pin it down. We finally asked the dean to describe the role function 
of the professional. He started describing him physically by saying, "He is an individu- 
al who has a crew cut, tweed jacket, tan trousers, and saddle shoes. " And there the 
dean sat with a crew cut, tweed jacket; tan trousers, and saddle shoes. He had a list 
of ten students whom he rank -ordered, and we predicted the rank order from the psycho- 
logical tests. He said this was great, and asked us how we did it. We said all we did 
was to assume from our earlier clues that he was a very narcissistic character who 
would only select people in his own image. So we had a very simple problem, which 
was to study him and figure out the kinds of individuals that get along with him. And 
interestingly enough, in a follow-up study, the factors that were relevant for predicting 

success m this situation were not the critical factors for predicting success in the pro- 
fessional career . 



— ~' YeSs this 1S wh y a lot of criterion analysis is not pure and does involve 
lifting yourself by your boot straps. And somehow we can’t escape, I’m afraid, some- 
thing that psychologists don’t like to do, and that is a rational analysis. Let me give 
you a concrete example. Could you, if you were asked to, set down some of the pre- 
requisites for the job of collecting garbage? Now there are three ways that I can think 
o going about answering this question. You can observe what garbage collectors do 
But it s conceivable that they might do the wrong things, I mean in my hypothetical 
case. They might pile the garbage in the middle of the streets or they might dribble it 
along as they go. Somehow we don’t feel that they are doing their job, even though this 

1S , ™ ay *** understand il - We could even go to people in the city government who 
should know and ask them what characteristics garbage collectors should have. And vet 
somehow, we feel that there is a kind of rational analysis of the role of garbage collector, 
qui e independent of any of these behavioral understandings of it. That is simply to set 

the garbage out of town. I don’t know if we can do that for the role of a scientist. It 

involves an understanding of what science is and what the ideal scientist is. I don’t 
mean asking a group of scientists whether they should have imagination or something 
like that. What are the things that it has to have or else it isn’t science any more? 

C- -Merton listed two of the requirements as disinterestedness and limited com- 
munism (sharing and divesting of personal involvement). And he had this truth element 
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in it. There were two or three others. 

C- -That's functional analysis. That is, if it does not have these characteristics, 
then it is no longer science. In other words, if people lie, somehow this violates our 
meaning of the word science. This is the way you get your criteria. This, you see, 
is quite apart from anyone's understanding of the situation. This is what bothers psy- 
chologists, because we never do things that way. 

Q--Well, doesn't this describe merely the minimum requirement? 

C--It is only part of the criterion analysis: we have to do other things, too. 

C--In our research, we had a category called potential creativity . These individu- 
als fit what we called psychological prerequisites for creativity. We are tabbing those 
things in the hope that we can follow them up in the future, to identify people whom we 
regard, on psychological bases, as being creative but who are not manifestly creative 
on die job. What we would hope to do is to be able to use the same variables in more 
controlled laboratory experiments . We would select subjects in terms of these variables 
and put them into a controlled set-up and see what they come up with. These aspects of 
potential creativity include all the psychological correlates of the functional analysis, 
but are present in individuals who do not have the proper environments for manifesting 
creativity. 

C--Now this is really pretty, because you are not really lifting yourself by your 
boot straps anymore. It alleviates the whole problem of intermediate criteria. The 
psychologist commonly shrugs his shoulders and says, 'Well, after all, in order to be 
a scientist you have to go through school, ” so we use school grades as the proximate 
criterion even though we know it isn't any good. But, if you would stop to think about it, 
its conceivable that a real theoretical analysis of the criterion - -of what it takes to be 
a scientist --would ultimately force the schools to change. That would break the vicious 
circle of simply reinforcing the status quo. 

, C--This is an important philosophical issue which we ought to face more when 
we deal with criteria: Should we aim at a criterion such as exists now, having people 
judge as they do, or should we as psychologists tell other specialists what kind of ideal 
characteristics the person or the criterion ought to have? 

Q--But then can we decide what an ideal scientist should be? 

C--I think that we can say a few things: for example, the natural scientist has 
got to have intense interest in the physical universe . Secondly, he has got to have a 
high degree of perseverance. 

C--That isn't the way that you do it. That isn't a functional analysis. You're 
probably right - -don't misunderstand me — but it isn't a self-evident truth that a per- 
son must be persistent in the same way it is a self-evident truth that he has to tell the 
truth . The distinction is between a functional sociological analysis and a psychological 
one. As soon as you get to perseverance you are on the psychological level. 
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C--Do you regard the results of functional analysis as having the status of postu- 
lates or of hypotheses? 

C- -Postulates. I would regard persistence as a hypothesis because at least some- 
body could question it. What a man produces may be science, whether he persists in 
such production or not . 

C- -Regarding truth -telling, I can envision a scientist being extremely biased 
(which has a shred of untruthfulness) and yet being a perfect scientist. 

C--Yes, you don’t inquire, ’’What is a scientist?”; you inquire, ’ What is the 
nature of science?” One element is the way in which one tests knowledge, what one 
means when he says, "I know. " If you say, ”1 know" as a scientist you mean some- 
thing different from what you mean when you say "I know" in general. You mean knowl- 
edge obtained in a certain way, in which a certain degree of confidence can be stated as 
the result of specified operaf'^ns. 

C"But as a scientist all you have to do is to affirm that you will so act. Further- 
more, the importance of this business about telling the truth is that science is free of 
any individual’s telling the truth. It does not matter if any single person tells the truth, 
because the method itself guards against the acceptance of falsehoods. 

C--The most important thing to be said about a person who says that he is a 
scientist is that he has a conception of some sorts of things which he will affirm as 
being the kinds of acts that he will do, and they will include all the things that are 
essential in order that his behavior will be called scientific. He has to accept methodic 
doubt; he will tell the truth as far as he can in all of bis observations, etc. But beyond 
that, I think that the range is extremely wide as to what kind of things a scientist can be. 

Q ~ " Is this th® approach to the criterion problem or just one approach? Do we 
have to build a set of postulates and from that derive various hypotheses which are 
then to be tested? 

C--It is just one approach to the criterion problem. You set up a certain set 
of hypotheses - - what sociologists call one of the functional prerequisites of the job of 
being a scientist. I think that you could come out with four or five. Parker and Merton 
have already done it. And then the next stop, if you wanted to predict this part of the 
criterion, would be to say, "Okay, as a psychologist, how can I measure these things 

in people? ’ Well, this doesn’t finish the criterion problem by any means --it is only 
the beginning. 

C--One of the things that sociologists do, which I think our role of psychologists 
forces us not to do, is to speak of "role" as an institutional requirement that must be 
fulfilled. An individual is put into the role or finds himself in the role. They don’t 
speak enough, at least from my point of view, of how to gain a role. By that I mean 
the factors necessary to gain the opportunity to be creative. Here is where the truth - 
lying item comes in. That is, in the role of scientists you don't lie, but you sometimes 
have to forget the role in order to get a chance to be the scientist. 



C - -You have the transition problem. You may have negative transfers. 

C --Check! This is well illustrated by one of the physiologists in our university, 
for example, where, in order to get research funds, you had to get the department s 
approval for your research study. A certain chap came in one day and presented his ^ 
research proposal. The department refused to approve it because they said it couldn't 
be done. He turned to them and said, "Gentlemen, I’m not asking you whether it can 
or cannot be done; I am asking you to support this. As a matter of fact I have already 
done it. One of the first things I learned on coming out of school was that everyone told 
me things could not be done. So I developed the technique where I do it first and raise 
research funds to support the study, but use them to do another study which people say 

cannot be done." Now I think learning these gimmicks is untruthful. But gaining the 
opportunity by having certain personality characteristics that are involved in the transi- 
tional phase is very critical, and sociologists don’t deal with the matter too well. 

C -There is a theoretical article on this, but I agree with you that nobody has 
ever worked at it . 

C--I think that should be said about functional analysis in general. The functional 
analyses" I know of are all ex post facto, many of them based on one anthropologist’s 
report regarding a primitive tribe . And they are plausible explanations of what took 
place. They had not yet, however, reached the point of trying to predict what would 
happen in the given situation in our own culture, so that we could check on whether they 
are postulates or whether they are as sound as they seem plausible. 

C— I think it would be a foolish pursuit to try to nail down any one single criterion 
of scientific creativity. I’m convinced that we are going to be faced with a multiplicity 
of criteria. Depending on the setting in which you are located, you will have any of a 
half dozen different criteria, depending upon your particular function in the social struc- 
ture. A number of people can be creative in different ways. The man who is a research 
director may be creative, but certainly his contribution will be of quite a different order 
than that of the man who is actually doing the work. And I think we should be prepared 
to deal with this kind of complexity. 

C-*Td like to raise two criteria which I think are more nearly psychological than 
sociological or philosophical. One is the criterion of products. One out of three students 
who enters our graduate school completes it. Of those who complete it, only 1 out of 3 
writes more than about 3 papers in a 9 -year period. That is, only 1 out of 9 individuals 
who enters a graduate school like the one to which I am referring, produces enough to 
be called productive. This really is less true in the physical sciences, such as physics 
and chemistry, but it’s very clear in biological sciences, social sciences and humanities . 
One might very well be able to define a minimum criterion of productivity below which 
one would say the likelihood of this individual’s being creative is pretty remote, simply 
because he isn't communicating with anybody, other than perhaps his students. So I 
would like to suggest just as one proximate criterion, something in productivity of pub- 
lished papers. You can do this more in some fields than in others. 

Another criterion is the judgments of colleagues. In one study we had two 
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independent panels of chemists select a group of chemists that they claimed were out- 
standing and creative. The agreement was quite tremendous, even though thi3 was done 
independently. . Further, when each chemist selected was approached to participate in 
this program, not one suggested that the panel should have picked someone else. We 
also did this with mathematicians. My point is that the judgments of colleagues might 
be collected in a fairly simple way, and I think it is a lot easier in chemistry and math- 
ematics than in many other fields. I think the judgments of colleagues can be used as 
a criterion of either the productive or the creative - -call it what you will — scientist. 

I’m suggesting that oniy a small proportion will get anywhere near being productive; 
even fewer, creatxve. 

Q- -How did the ratings correlate with your productivity indices? 

C--I think quite high in general. 

Q- -What did they rate? Productivity? 

C- -Creativity, ana we added some definition of creativity for them. I think that 
a rating on productivity would have been just about as good. 

Q- -Did they rate the man or did they rate the things he did? 

C--They did both. They were asked to do two kinds of ratings. One was ' /hether 
the ideas or the things he has created have affected the field, to the extent that there is 
no doubt that he has really altered the field. The other was that in his ways of working 
and the ways in which he proceeds to do various things he seems to be unusually creative. 
I think there are better ways of doing it than the particular lines we used. My main 
point is that the judgments of others can be used. 

C--I must say that I have had experience with studies doing this. I tend to take 
a strongly negative view of those suggestions. When you deal with socially recognized 
productive acts, you’re missing a great many people who are highly creative, because 
there are many creative people who don’t produce. I think that what you have to do is 
get some way of eliciting performances from people in general of such a nature that 
you cr n rank -order them in terms of creativity. That is, at this point, you have to 
keep the motivational factors out of it as far as possible. You want to give them every 
opportunity and as many different media as possible because there may be a self- se- 
lection of media also to express what originality they have. 

Q - -As a criterion , though? 

C--Yes, as a criterion. If you have a high degree of reliability in judging whether 
a performance has occurred or not in a creative manner, you will get psychological in- 
sight into the process of creativity, because you will have eliminated so much of the 
error that is involved in the other two kinds of evaluations. We studied 80 graduate 
students who were reaching the Ph. D. level and were working on theii dissertations. 

We had faculty members rate them on originality. Then we had the psychologists, on 
the basis of 3 days of observation of their behavior, rate them on originality. We next 
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took a large group of free -response tests like mosiac construction, response to ink blots, 
etc. , and let persons who had never seen the subjects rate them. In analyzing the dis- 
crepancies among these 3 sets of ratings, one thing became pretty evident: that it is so 
easy lo mis rate a person on the basis of personal bias. For example, an unlikable person 
is almost always misrated as to his originality. Certain kinds of hostility, not necessarily 
just unlikableness, but certain kinds of manifest hostility, tend to get the person misrated. 
On soundness there is a terrific preconception, somehow, that a person who simply 
doesn’t have sound judgment is not original. And this persists even when psychologists 
are supposedly guarding themselves against it. 

C --Aren't you talking about two different stages? One of you is talking about 
graduate students and the other about mature scientists who have had their Ph. D. or 
about 10 years. What do we mean by ’’criterion?" Also, you say all you do is determine 
whether the behavior occurs or not. But who decides what behavior to look for? There 
is a judgment here. 

C--Well, in a simple game like anagri^s, for examnle, we can tabulate the 
frequency of responses given in a certain population, then check each answer in a dic- 
tionary. That’s creativity in these test situations. 

C--Yes, I think you’ll get much less error finally in your criterion variables if 
you deal with performances of this sort, where motivation to be socially visible as 
creative, for example, does not enter in. 

C--Some other kind of motivation does, though. Your test situation involves a 
very special kind of motivation. It may be that what we mean by social creativity is 
the ability to be creative given a certain social motivating condition . I would say that 
part of the criterion analysis has to include the motivational element in the job situation. 
For example, a psychologist had two performance situations, in one of which the trainees 
were told that as soon as they finished a job they could quit. Another group was told that 
it was somehow, a test of their ability. Now, performance under those two motivating 
conditions was pred ated by different motivational scores. How can you expect to suc- 
ceed if you’re trying to predict the performance under a variety of motivating conditions 
from any given predictor? You have to know what the motivating conditions are in order 
to get O ood predictions from your predictors . 

C--As a matter of fact, our high "need achievement" people do worse under test 
conditions than when they're working on their own. If you put them in a competitive situa- 
tion, they do less well. The prediction is that they'll do better on a desert island or when 
they're not being asked to do something. 

C--What we're trying to predict is how people will perform in the world as it is. 

C--Yes and no. We're trying to do both. We write the equation in terms of what 
the conditions are that we're trying to predict, and all I'm saying is that your criterion 
analysis has to include the situational variables, including motivation. When you do your 
criterion analysis, you include the motivational variable there. Also include it at the 
time of the test analysis. Then, if the motivational conditions during the test analysis 
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are similar to what they are in the criterion analysis, your prediction will be better. 

I dcfn't think this is minor variance, by the way. 

C--I look at it somewhat differently. Your intra-test criteria of originality car. 
be congruent in certain individuals with social creativity. But in others, psychological 
criteria may be high and social criteria low. You may have several patterns, all of 
which may lead to creativity. I think we overlook the fact that actually when we as 
psychologists develop the tests, we are part of the new criterion. 

C— I can think of examples in our organization of the complexity of the motiva- 
tional situation for scientists. 

— Yes, the motivational conditions in the actual criterion situation can be very 
complicated. But I don't see any alternative to making analyses of these criterion situ- 
ations in terms of the current motivations of various sorts, including frustration. May- 
be in your aptitude tests you ought to interrupt these people half-way through. 

C— It depends upon whether you want to predict whether a person will produce 
original acts or recognize them as such. 



CRITERION COMMITTEE REPORT 



Reported by 
L. R. Harmon 



S--The outline that follows will pertain solely to the domain of research scien- 
tists. That was deemed the most feasible and profitable point of attack; for this confer- 
ence; reserved for later development were questions of criteria for applied scientists, 
teachers, and administrators of science. Each of these domains involves unique prob- 
lems, and time permitted attack on only one set. The general outline r *i' procedure which 
was developed is, however, applicable to ail. That is the procedure of working back- 
ward from some agreed-upon ultimate criterion to some more immediate and feasible 
working criterion which ,an serve as a reasonably good substitute. 

Our ultimate criterion is a measure of the individual scientist's total creative 
scientific accomplishment. We exclude here his non -creative accomplishments and 
non-scientific creations. These might be important considerations in some other do- 
mains, but not within the domain which we are now considering. 

\ 

For the assessment of an individual's total creative scientific output we would 
require a panel of scientists for each field of specialization -- people competent to judge 
the creative merit of scientific accomplishments. The first task of the panel would be 
to set up an objective evaluation schema by which to judge the products of the individual 
scientists. For it is definitely the products, and not the scientists as individuals, that 
must be judged for criterion purposes . 

It is obviously not possible to develop such an ultimate criterion for all research 
scientists. For living scientists, the full measure of accomplishment is of course 
seldom available. Furthermore, the development of objective measures of creative 
scientific output is an onerous task and one which is feasible only on a sample of all 
individuals within any particular category. For the end purposes for which criteria 
are desired, it is not particularly relevant, either, to assess the products (and thus 
indirectly evaluate the individuals) of scientists long departed. What is required, there- 
fore, is a sample of contemporary scientists whose total life's creative output is largely 
complete, so that errors of approximation of the ultimate total output will be reduced 
to a minimum. For example, scientists aged 60 years might be included. 

The first steps in the development of this ultimate criterion are somewhat in- 
direct and involve what may be criticized as circular reasoning. First, one must se- 
lect a pc. lel of scientists who are in a position to evaluate another sample of scientists. 
This first step must, of course, be taken by whoever is developing the criterion- -pre- 
sumably a psychologist in this case. The panel would be asked to select for evaluation 
scientists within their own field who have manifested a wide range of scientific creativity. 
They would then be asked to place in rank order the members of this evaluation sample, 
according to their best judgment of the members’ creativity. The reason for this highly 
subjective step will become more apparent as we proceed. It will provide a check on 
another kind of subjectivity - - that of setting up an evaluation schema for the rating of 
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products, and of the weighting system for elements of this schema. 

For the members of the evaluation sample, there would then be assembled the 
necessary information regarding their total scientific output. These products would 
then be evaluated according to the objective evaluation schema mentioned earlier . The 
procedures for evaluation cannot be specified in detail. What is important is that means 
must be employed to insure that, at this stage, products, not people, are evaluated. 
Anonymity of products, insofar as possible, would be one such means. Furthermore, 
to avoid individual and sub-cultural biases, techniques such ais those described earlier 
by McPherson would be employed in the product evaluation. 

When all products of the sample had been individually evaluated, the problem 
would arise of the weighting schema to use in referring these evaluations to the individ- 
ual scientists . Should a simple sum of individual product weights be used, or should 
some more complex combination schema be employed? It would be necessary to de- 
termine some method for this process and to take pains to insure that the weighting 
schema reflected what the panel considered to be the proper relationships, i.e., that 
no statistical artifacts were inadvertently introduced. 

In all these procedures, the role of the psychologist would be to help the special- 
ists in each field to achieve their goal of inclusiveness, objectivity, and proper weighting 
of criterion elements. The psychologist does not here set up the criteria. He helps to 
make clear to the specialists just what it is they are doing, and helps in verification of 
what they have done . 

When the products have been individually evaluated and the weighting schema 
devised, all of the products of each scientists would be combined to yield a single cri- 
terion score. This would yield a series of scores applied to the individual scientists 
which could be compared with the rank -ordering process accomplished by the panel as 
its first step. When this comparison had been made, the panel would be asked, "Does 
this evaluation represent what you had intended to accomplish? If not, wherein does it 
err, and in what directions are corrections to be made?” The corrections would have 
to be objective procedural ones, so that the evaluation procedures themselves could be 
revised to yield the kind of results that scientists could agree upon. For example, if 
the procedures placed high on the scale some individual who had turned out a large 
number of mediocre papers, but none o f outstanding significance, it would probably be 
agreed that the procedures needed revision. This check, then, while somewhat circular 
and subjective, is a necessary one to insure ultimate objectivity. 

So far, we have described what we have called an ultimate criterion, or a close 
approach to one It is not a practical criterion for our purposes because of its tremen- 
dous time lag and the great expense involved in deriving it. It is necessary to arrive 
at some intermediate criterion, i.e., one which can be arrived at earlier in the scien- 
tist’s life span. It is also necessary, simply for reasons of economy, to try to develop 
substitute criteria -- those which will predict the ultimate criterion -- but which may be 
8 simpler and less costly. Obviously, the practical working objective is an early inter- 

mediate substitute criterion, which would in turn correlate highly with the ultimate cri- 
terion. Thus, an ideal simple criterion would be developed early in the scientist’s 
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career which would predict accurately, or correlate highly with, the ultimate criterion. 

It should be noted at this point that essentially objective statistical research 
means might be used for deriving intermediate and substitute criteria from the original 
or ultimate criterion, but that the latter depends finally upon subjective human judgment . 
There is no escape from this matter of human judgment. We may only choose how it 
is to be exercised, and how known types of bias in judgment can be counteracted. Our 
choices have been to control as far as possible the kind of bias which might be called 
the "how I feel about you" kind by making product judgments annonyrrously, insofar as 
possible. Individual biases with regard to particular kinds of products and some de- 
gree of sub -cultural bias will be controlled, in part at least, by multiple judgments and 
by having panels representative of as wide a variety of sub -specialists as possible. 
Furthermore, schema of evaluation will be set up which would tend to objectify judg- 
ments of individual products . This latter step pushes the subjectivity back to what is 
involved in the schema; the result of this step is to bring it into the public domain, and 
hence into a broader sphere of inter-subjective scrutiny. What we have tried to do, in 
short, is to define and identify the role of subjectivity, rather than to pretend that we 
have abolished it. 

This is the general outline which we propose to follow. Thus far, all we can do 
in filling in details is to suggest some possible ones. For example, the products that 
would be evaluated would include papers and patents. When they are evaluated, an 
historical study would be necessary in order to achieve proper attribution. (We are 
concerned here with cribbing by one person from another, disputes over precedence, 
and so on. ) In evaluating papers, we might start with a crude count of papers published, 
or with number of citations . This first crude figure might be improved by evaluations 
of the quality of the papers. Several stages of refinement might be tried in order to 
determine how far it is necessary to go to achieve stability of rank orders. For ex- 
ample, a brief consideration of titles alone might be made, or the panel might pass 
judgments on the basis of abstracts of the papers, or, finally, evaluate the entire docu- 
ment. Similar procedures might be used for patents. It is apparent that step-by-step 
elaboration of these procedures would throw light on the extent to which substitute cri- 
teria might be successfully employed. 

Further suggestions for substitute and intermediate criteria include such things 
as rate of achievement. That is, the age at which any given level of creative scientific 
achievement was reached might be considered, somewhat after the manner of an intel- 
ligence quotient. Another substitute criterion might be membership in scientific organ- 
izations and officer ship in such organizations. We would look here, however, for the 
possibility of suppressor variables. For example, prominence in an organization might 
be achieved by means other than creative productivity. We might refer to it as an 
"operator" as contrasted with a "producer" variable. If prominence in organizations 
were found to correlate highly with the rank order of scientists as judged directly by a 
panel of their peers, but to correlate low with the creative productivity criterion, it 
would be a valuable suppressor. The position of an individual in an institutional hier- 
archy would be similarly evaluated. We might conceivably find here a measure that 
correlates low with the creativity criterion, but positively with a teaching or adminis- 
trative criterion. In accordance with our aim of considering one domain at a time, 
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position in an institutional hierarchy would then be discarded, or, possibly, used as a 
suppressor depending on its statistical characteristics, in development of a scientific 
creativity criterion. 

Finally, the simplest and most direct of the substitute criteria is direct evalua- 
tion of scientists — ratings by other scientists . All experience indicates that such judg- 
ments would have to be made by panels representing a wide variety of sub -fields, in 
order to cancel out sub-cultural biases as well as possible and in order to minimize the 
effect of individual personality factors. As with any of the other, less direct evaluations 
of men via position, salary, etc., the efficacy of this substitute criterion would have to 
be judged by its correlation with our ultimate criterion of creative scientific productivity. 
The model for any combination of intermediate and/or substitute criteria would be that of 
the multiple correlation with the ultimate criterion. 

The third area of criterion development is one which comes earliest in one’s 
career, and so meets one of the most important practical requirements. It is, however, 
the least satisfactory conceptually, as it becomes impossible to draw a clear line dif- 
ferentiating it from predictor variables. This area consists of judgments of perform- 
ance, of behavior related to scientific activity. In this area, we have come up so far 
with three rather unrelated points which are of importance for criterion development. 
First, we are concerned with a typical mode of functioning as a scientist, rather than 
with unusual examples. The second point concerns techniques: we might seek to meas- 
ure the amount of time and effort spent in creative activity. Third, we might well em- 
ploy panel ratings of performance of observed samples of activity. If this is to be done, 
we would necessarily have to do some standardization of the behavior samples to be 
observed. At about this point, one must agree that we are passing from criterion be- 
havior to test behavior, or predictors. A person is given some sort of problem to solve, 
and judged on how he solves it. This is one definition of a test, and while it may, if we 
are fortunate, correlate well with an intermediate or even an ultimate criterion, it can- 
not be considered conceptually as a criterion, but rather as a predictor. 

It is apparent that we have given least attention to the third field - -that of per- 
formance evaluation. This neglect is partly because of time limitations, but mostly be- 
cause it is farthest from our assignment, the criterion, and verges over into the test 
areas that other groups are to consider . 

There are a number of areas in which we have left unsolved problems. One of 
these is the definition of the criterion according to the function to be served by it. That 
is, the actual criterion scores or rank -orders assigned must vary according to how the 
criterion is to be used. For example, a criterion used as a means of test -selection, 
to build up a battery of tests for the screening of personnel, will be a different criterion 
from one used for promotion of personnel already on the job. One must consider in this 
connection such elements as the inter -correlations of criterion elements, their reliabil- 
ities, the extent of individual differences exhibited with regard to any criterion element, 
and so forth. The work of H. E. Brogden, E. K. Taylor and R. H. Gaylord is pertinent 
in this connection. 

Another domain in which we have not formulated any approach and have only 
some scattered comments concerns criteria for teachers. For these people the products 



are people, or more precisely, behavior changes in people who have come under their 
influence. A teacher may create by stimulating creativity in a student. The creativity 
of the teacher would not, however, be registered on the criterion which we have de- 
scribed. Two modes of evaluation of teachers follow by analogy from the procedures 
outlined above for research scientists: one might evaluate the creative productivity of 
their students, or one might evaluate the teachers' modes of performance. It seems 
highly probable that these two approaches would yield quite dissimilar results, that 
the correlation would be low. 

Finally, we have just touched upon evaluation of administrators in science. We 
have no formulations, but might suggest that considerations in this domain might paral- 
lel to some extent those for teachers. Administrators might develop and encour ag e 
creativity in individuals, might bring about conditions in which people became creative, 
either individually or in teams, or they might inhibit such developments. Their mode 

of performance, too, might be investigated, as well as the productivity of those under 
their administration. 

-Could 1 ask why you left out all of Stein's treatment of the criterion problem? 
Maybe it isn t relevant, but you know the whole problem we discussed the other night 

of the ideal performance and the dangers you get into of just reproducing your cultural 
biases. 



C--Ls it not quite conceivable that a man could fill this role almost perfectly 
without creating anything? And is not the reverse also true -- that is, might he not 
create a lot of things and still be a Nazi? 

C— In the role configuration that I presented the other night, a man would not 
have tc be creative in order to fill the role. I still haven't worked out all aspects of 
the system or of the individual which would permit him to rise above this role. I know 
he must fulfill the role, but he needs X-plus. But what the ’'plus" is, I’m not absolutely 
certain yet; that is what the research is trying to determine. 

C- -There is a man at my institution who is one of the most creative people I 
have ever seen, but he wouldn’t fit any role that I could possibly imagine for a scientist. 

“You mean he lies and fakes his data and all these things? 

2 “No, he can t be a scientist if he fakes his data; he doesn't even belong in that 

category. The role configuration doesn't distribute these people at all; it just says which 
pec pie are within your category. 

C--I really believe it’s a serious error to think that social consensus is your 
final criterion. Social consensus, for example, as to the value of certain inventions 
or patents may be a purely temporal thing. And if you're trying to predict that, what 
you need to know is the social psychology of the judges. For example, if you're trying 
to predict the behavior of the person getting a patent, then you need to know what the 
characteristics of the behavior are. Now, whether or not this particular outcome be- 
havior is judged by peers or anybody else as good or bad, or more valuable or less 
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valuable, is quite a difficult thing. It seems to me you are stuck with a social con- 
sensus there, and you don't want it. 

C--I honestly don't get it. As I understand it, what the committee is proposing 
is that they recognize their need to have a set of criteria developed by specialists in a 
given field and applied to products . They don't want us as psychologists to say what the 
elegant criterion is. That's why they’re going to these judges, these peers in the field, 
to get the criterion schema. 

C - -The question is, what is the characteristic of the behavior you are trying to 
predict? As a psychologist you can describe that. You can also get a panel of judges 
to say whether it's valuable behavior or not. Now, which do you want to do? 

C--We were not having our judges judge behavior. They were judging products, 
and that's quite different from behavior. 

C--The third portion of your criterion concerned judgments of performance, and 
I say you ought to leave that judgment out. Physicists don't try to make estimates of 
how hot it is. They stopped that a hundred years ago. Is the best estimate of how hot 
it is the pooled judgment? Better to use a thermometer, and get away from the social 
psychological problem. 

C--We are all quite willing to look at a whole series of behaviors that we as 
psychologists have dreamed up and wish to observe, but we v/ill finally ask about these 
behaviors, are they related to productivity and quality? That is our point. Our proce- 
dure would screen out very shortly those that are not. From the standpoint of our dis- 
cussion, the question is whether the behavior we are observing is within the definition 
of criterion or predictor. 

C- -I'd like to suggest an addition to the judgment area of the criterion - -the 
work at Ohio State by Mooney and others . They have devoted a lot of time to develop- 
ing indices of creative behavior which would describe the creative person. They have 
several hundred items grouped into four dimensions. The items are amenable to check- 
list, self-description, or rating format. I think that an important part of the judgmental 
criterion is the estimates by peers - -by scientists of each other --in terms of daily be- 
havior that is deemed creative behavior on the part of the Ohio State group. As far as 
I'm concerned, these should be considered independent of any kind of creative criteria 
where you can count the number of products one puts out, etc. That is to say, I would 
not want to have judgments of these indices of creative behavior stand or fall upon the 
degree of correlation they hfve with criteria such as productivity. I would want them 
to stand on their own as judgments of a fellow scientist based upon indices a psychol- 
ogist has brought out of his head. 

Q- -But how do you know you are describing creative behavior unless you have a 
criterion to put it up against? 

C--This is the criterion; it is coming out of your head. 
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C--Well, we just can’t accept that as a definition of criterion. It may be a good 
predictor, but it isn’t what we are talking about. 

C--But we still have a problem. When you go to a specialist, you must not let 
his concept of what is soundness in a product interfere with his concept of creativity. 
For example, in one of John Arnold’s problems he calls in a jury of engineers to rate 
designs. He finds that the correlation between an area called ’’engineering soundness" 
and an area called "idea” is high. When something of this kind happens, you as a psy- 
chologist must assert yourself strongly, and help the specialist define creativity as 
disparately as possible from soundness . In other words, an idea or design or chemical 
innovation can be extremely Rube Goldbergish, and from a soundness standpoint within 
the discipline concerned, it can be terrible. But it may be creatively important. 

C- -Right. In patent law they have a section for silly inventions, recognizing 
that today’s silly invention is tomorrow’s marvels 



C~The point is that the psychologist cannot play an entirely passive role. You’ve 
got to worx to keep these things such as soundness and originality as orthogonal as pos- 
sible, and not let large terms with halo effects merge the elements of your criterion. 

s get back to Stein’s point. He asked, "Whom do you expect tc have the 
highest aspirations, those rated high creative or low creative?” We said the low cre- 
ative ones. He said, "That’s right. ’’ So the low creative persons may have long pro- 
jects germinating which deny them constant publications but, on the other hand, consti- 
tute high aspirations. They may be working on some terrific projects. In terms of the 
indices of creative behavior I have in mind in the Mooney dimensions, you’d be getting 
at the behaviors which typify the fellow with the high aspirations and the desire to crack 
some real soul-searching problems. But when you come along, he may still be rated 
low creative because he hasn't cracked it yet. 

C All right, but you re not talking about what I’m talking about. If we have an 
ultimate criterion, and he doesn't crack it, he isn't creative. 

C— I still don’t like your strict opportunism in the approach to this behavioral 
area. You throw something out because it doesn’t correlate with the assessments of 
the product. Let’s take one of the Ohio items: "He is open to all of his experience. " 

This is an aspect of creative behavior — openness to one's experience .in the visual area, 
auditory area, kinesthetic, or what have you. I can envic ion people being rated high on 
this kind of thing; yet perhaps the product might be rated low, and you wouldn’t find any 
correlation. I m just saying that this is another part of the criterion situation. 

2-How can you call this a kind of creative behavior if it doesn’t produce anything? 

C--I guess what’s bothering me all along is that I object to the title of this confer- 
ence, because "creativity" is an evaluative term and this is a social -psychological prob- 
lem. As psychologists, we should simply predict different types of behavior. If a cer- 
tain job calls for behavior of "Type A, " we’ll put that person in the situation. I think it 
is really a little misleading to talk about "creativity. " You can forget that term altogether. 



Indices of creative behavior such as the Ohio State group is looking for may not relate 
to productivity. When sheer quantity pushes up a person's score, and quantity is a part 
of a composite score. . . 

C--But we're interested in quality too. If that were low, the individual’s product 
would be rated low. If the first procedures resulted in a high producer of mediocrity 
being rated at the top of the list, we would ask the jury, "Is this what you want?*' and 
they would say, "No, let’s revise it. " And we'd proceed to go through the thing again 
until we came out where we were supposed to. 

C— I agree. I cannot personally think of behavioral variability as ultimate. It's 
the effects of the behavior that you have to judge. Take the example of the teacher. 
Using the behavior of the teacher - -the teacher's performance as observed - -what is 
the ultimate criterion? The ultimate criterion is what this behavior produces in the 
students. There is a well-known example of a physics teacher who puts on a wonderful 
show. His performance, as judged by the way he conducts his instruction, would be 
rated pretty high by almost everybody. But you will search in vain for any Ph. D< 's of 
his that have ever amounted to anything. There are a lot of similar examples. 

C— You brought up a point that I think you skipped over a little too lightly. It 
should be worthy of a few refinements. That is the "operator" or "institutional" type 
of person. He is not necessarily as bad as you think he is, provided he can be identified 
along one of two lines that Carter has identified in describing the dimensions of behavior 
in groups. The two dimensions are "goal facilitation" and "individual prominence. " 
Now, if a person is operating to facilitate the goal of the group, if he's operating to pro- 
tect the group, then give him credit for creativity. But if he’s operating along the other 
dimension, individual prominence, then you've got a suppressor variable to deal with, 

I think this refinement should be introduced into that cluster. 

S--What we meant by "operator" was the individual prominence sort of thing. 

C--It's important to look at this other kind of operator, too. He's needed, for 
example, in military research organizations. You need an intermediary with the mili- 
tary who can speak their language. Sometimes your lay researchers don't have the 
language --it takes an operator. It may be a highly creative performance of a social 
nature. 



S--This would come under the administrative portion of our over -all criterion, 
and we haven't done much to develop that. 

C— I’d like to get back to the problem of "creativity" in the criterion. There 
is an aspect that has worried me right from the start. That is that cur concept of cre- 
ativity has within it two contradictory elements. First, the notion that creativity is 
original, or unhabitual or something of that kind, and second, that it must meet some 
kind of consensus of approval with respect to something like beauty or utility. I think 
that this is a dangerous proposition. It introduces a kind of evaluation that makes it an 
awfully slippery thing to handle. What are you going to do about the paranoid who evolves 
a wonderfully ingenious, elaborate and integrated delusional system? Is this an act of 



creation, or isn't it? Now, for my money, it is a highly creative act, ahd I don't think 
We ought to say that this one doesn't count because we> can't use it for anything. 

S--No, utility has never been suggested as a part of the criterion. 

C - -What you'd be doing is selecting creative paranoids, that's all. 'When we talk 
about intelligence, we don't require that the individual behave intelligently in order to say 
that he has intelligence. We are perfectly content to attribute intelligence to a man who 
behaves stupidly, and to say that this man has the potentiality - -the inner psychic capac- 
ity -- for performing these mental operations, whether he does perform them or not. 

1 : • * , * t » *’ * - 

C--But you deckle this on the basis of tests that do correlate with originality in 
people who displayed it. We require a criterion for selecting the test or predictor, not 
for direct selection of the individual scientist. 

C- - What I want to get at is this: we often recognize that creativity may be thought 
of as a potential that is not necessarily realized in any actual creation, in the same sense 
that intelligence can be recognized as a potential of an individual, even though for neurotic 
or for other reasons he behaves stupidly in particular situations . 

C- - Well, I think that our problem is that we are saying first of all that a person 
is not creative unless he creates, in terms of his ultimate criterion. Once we have that 
ultimate criterion, we should be able to find behaviors that are associated with it, or 
which cause or produce creations. Working backward from the ultimate criterion to 
intermediate criteria, substitute criteria, we arrive at predictors which we hope will 
identify potentially creative individuals at very early ages, long before they create. But, 
to do this job, We have to relate these behaviors to a criterion, just as you have to re- 
late your intelligence test to some criterion outside the testing. " Where we got into 
trouble before was in considering a series of judgments of originality and a series of 
tests of originality, and simply relating, the two. The thing is circular. 

C- -Granted, but if you get fat enough consolidated circles, you've got a valid 
something --call it factorial validity. 

C--Xt is internally consistent. But it may not be consistent with anything outside 
its own system. The point is very ably summed up by the earlier statement that an an- 
alytical scheme is developed for each product and applied to all the prodtj&ts of every man; 
then all the men are ranked according to this analysis. If we get a good analytical scheme 
for classifying the products, then we might be really lucky and find that the behavior and 
the end product are not so far apart. 



259 



REFERENCES 



1. Adorne, T.W., Frenkel -Bruns wik, E., et al. The authoritarian personality. 

New York: Harper, 1950. 

2. Allison, Roger B. A factor analysis of measures cf learning. Ph.D. dissertation, 
Princeton University, 1955. 

3. Arnold, J.E. (Ed.) Creative seminar notes creative engineering lab. Cambridge: 
M.I.T., 1954. 

4. Barnett, H.G. Innovation: the basis of cultural change. New York: McGraw-Hill, 
1953. 

5. Barron, F. Artistic perception as a possible factor in personality style: its meas- 
urement by a figure preference test. J. Psychol. , 1952, 33, 199-203. 

6. Barron, F. Case study of a residual. In A. Burton &R.E. Harris, Case outlines 
in clinical and abnormal psychology , 2nded., New York: Harpers, 1955. 

7. Barron, F. Complexity -simplicity as a personality dimension. J. abnorm. soc. 
Psychol. , 1953, 48, 163-172. 

8. Barron, F. The disposition towards originality. J. abnorm. soc. Psychol., 1955, 
3, 478-485. 

9. Barron, F. Personal soundness in university graduate students . University of 
California Publications in Personality Assessment and Research, Oct., 1954, No. 1. 

10. Barron, F. Personality style and perceptual choice. J. Pers. , 1952, 20, 385-401. 

11. Barron, F. Some personality correlates of independence of judgment. J. Pers., 

1953, 21, 287-297. 

12. Barron, F. Some relationships between originality and style of personality. Insti- 
tute of Personality Assessment and Research, Berkeley: University of California, 

1954. 

13. Barron, F. Threshold for the perception of human movement in inkblots . J. 

consult. Psychol. , 1955, 19, 33-38. ~~ 

14. Bergson, H. The creative mind, New York: Philosophical Library, 1946. 

15. Bibliography on Creativity . Prepared by Creativity Sub-Committee of Research 
Personnel Committee Industrial Research Institute, Inc., 1955. 



creation, or isn't it? Now, for my money, it is a highly creative act, ahd I don't think 
we ought to say that this one doesn't count because we can't use it for anything. 

S--No, utility has never been suggested as a part of the criterion. 

C--What you'd be doing is selecting creative paranoids, that's all. When we talk 
about intelligence, we don’t require that the individual behave intelligently in order to say 
that he hat. intelligence. We are perfectly content to attribute intelligence to a man who 
behaves stupidly, and to say that this man has the potentiality --the inner psychic capac- 
ity -- for performing these mental operations, whether he does perform them or not. 

C But you decide this on the basis of tests that do correlate with originality in 
people who displayed it. We require a criterion for selecting the test or predictor, not 
for direct selection of the individual scientist. 

C--What I want to get at is this: we often recognize that creativity may be thought 
of as a potential that is not necessarily realized in any actual creation, in the same sense 
that intelligence can be recognized as a potential of an individual, even though for neurotic 
or for other reasons he behaves stupidly in particular situations . 

C- -Well, I think that our problem is that we are saying first of all that a person 
is not creative unless he creates, in terms of his ultimate criterion. Once we have that 
ultimate criterion, we should be able to find behaviors that are associated with it, or 
which cause or produce creations. Working backward from the ultimate criterion to 
intermediate criteria, substitute criteria, we arrive at predictors which we hope will 
identify potentially creative individuals at very early ages long before they create. But, 
to do this job, we have to relate these behaviors to a criterion, just as you have to re- 
late your intelligence test to some criterion outside the testing. Where we got into 
trouble before was in considering a series of judgments of originality and a series of 
tests of originality, and simply relating the two. The thing is circular. 

9 Granted, but if you get fat enough consolidated circles, you've got a valid 
something --call it factorial validity. 

C— It is internally consistent. But it may not be consistent with anything outside 
its own system. The point is ver” ably summed up by the earlier statement that an an- 
alytical scheme is developed for each product and applied to all the products of every man: 
then all the men are ranked according to this analysis, if we get a good analytical scheme 
for classifying the products, then we might be really lucky and find that the behavior and 
the end product are not so far apart. 



16. Bloom, B.S. Changing conceptions of examining at the University of Chicago. In 
P.L. Dressel, Evaluation in general education, Wm. C. Brown, 1954. 

17. Bouthilet, L. The measurement of intuitive thinking. Ph.D. dissertation, Univer- 
sity of Chicago Library, 1948. 

18. Brandwein, Paul F. The gifted student as future scientist . New York: Harcourt, 
Brace & Company, 1955. 

19. Brogden, H.E., Taylor, E.K., & Baier, Donald E. Experimental design: utili- 
zation of an unreliable and a biased criterion. Educ. psychol. Measmt, 1953, 

13, 27-33. 

20. Bush, V. As we may think. Atlantic Monthly, 1945, 176, 101-108. 

21. Bush, V. For man to know . Atlantic Monthly , 1955, 196, 29-34. 

22. Bruner, J.S. Personality dynamics and the process of perceiving. In R.R. Blake 
& D.V. Ramsey (Eds . ), Perception: an approach to pe r sonality. New York: 

Ronald Press, 1951. 

23. Catteli, R.B. Personality: a systematic theoretical and factual study . New York: 
McGraw-Hill Book Company, Inc., 1950. 

24. Clifford, Paul I. A study of the personality organizations of a selected group of 
highly creative chemists and mathematicians.- Ph.D. dissertation, University of 
Chicago, 1953. 

25. Cole, Charles C., Jr. Encouraging scientific talent. A report to the National 
Science Foundation . College Entrance Examination Board, 1955. 

26. Davis, Paul C. A factor analysis of the Wechsler -Bellevue intelligence scale, 
form I, in a matrix with reference variables. Ph.D. dissertation, University of 
Southern California Library, 1952. 

27. Davis, R.C., Mellinger, G. , Pelz, D.C., & Baumgertel, H. Interpersonal factors 
in research, part I . Ann Arbor: Institute for Social Research, University of Mich- 
igan, 1954. 

28. DeCharms, R., Morrison, H.W., Reitman, R. , & McClelland, D.C. Behavioral 
correlates of directly and indirectly measured achievement motivation. In D.C. 
McClelland (Ed. ), Studies in motivation. New York: Appleton -Centurv -Or ofts. 

1955. 



29. Devereaux, E.C., Jr. The sociology of gambling. Ph.D. dissertation, Harvard, 
1949. 



30. Duncker, D. On problem solving. Psychol . Monogr . , 1945, 58, No. 5 (Whole 
No. 270). * 

31. Foote/ N.N. &l Cottrell, L.S. Identity and interpersonal competence. Ghicago: 
University of Chicago Press, 1955. 

32. French, J.W. The description of aptitude and achievement tests in terms of ro- 
tated factors. Psychometric Monogr . , 1951, No. 5. 

33. Frenkel -Bruns wik, E. Intolerance of ambiguity as an emotional and perceptual 
variable. InJ.S. Bruner, & D. Kerch (eds. ), Perception and personality: a sym- 
posium . Durham: Duke University Press, 1949. 

34. Ghiselin, B. The creative process . Berkeley and Los Angeles: University of 
California, 1952. 

35. Ghiselin, B. The creative process: a symposium. Psychol. Book Rev., 1952, 2, 
46-49. 

36. Green, R.F., Guilford, J.P., Christensen, P.R., & Comrey, A.L. A factor - 
analytic study of reasoning abilities . Psychom . , 1953, 18, 135-160. 

37. Guilford, J.P. Creativity. Amer. Psychologist , 1950, 5^ 444-454. 

/ 

38. Guilford, J.P. Dimensions of intellect. In Deliberations, collogue internationale 
de Tanalyse factorielle et ses applications . Paris: Centre National Recherche 
Scientifique, 1955. 

39. Guilford, J.P. Some recent findings on thinking abilities and their implications. 
Inform. Bull. Train. Anal. Development. , 1953, 3, 48-61. 

40. Guilford, J.P. Factors in problem solving. ARTC Instr. J. , 1954, 197-204. 

41. Guilford, J.P. et al. A factor -analytic study of creative thinking. Rep. from 
Psychol. Lab. , University of Southern California, 1951-1952, Nos. 4 and 8. 

V 

42. Guilford, J.P. , Christensen, P.R., Kettner, N.W. , Green, R.F., & Hertzka, A.F. 
A factor -analytic study of the Navy reasoning tests with the Air Force aircrew clas- 
sification battery. Educ. psychol. Measmt, 1954, 14, 301-325. 

43. Guilford, J.P. , Kettner, N.W., & Christensen, P.R. A factor -analytic investiga- 
tion of the factor called general reasoning. Rep. Psychol. Lab., No. 14. Los 
Angeles: University of Southern California, 1955. 

44. Guilford, J.P.,, Kettner, N.W., & Christensen, P.R. The relation of certain think- 
ing factors to training criteria in the U.S. Coast Guard academy. Rep. Psychol. 
Lab. , No. 13. Los Angeles: University of Southern California, 1955. 




262 



45. Guilford, J.P. , Berger, R.M. , & Christensen, P.R. A factor -analytic study of 
planning. I. Hypotheses and description of tests. Rep. Psychol. Lab., No. 10. 
Los Angeles: University of Southern California, 1954. 

46. Guilford, J.P., Berger, R.M., & Christensen, P.R. A factor -analytic study of 

planning. II. Administration of tests and analysis of results. Rep. Psychol. Lab. 
No. 12. Los Angeles: University of Southern California, 1955. ~ ” 

47. Guilford, J.P. , Kettner, N.W., & Christensen, P.R. A factor -analytic study 
across the domains of reasoning, creativity, and evaluation. I. Hypotheses and 
description of tests. Rep. Psychol. Lab. , No. 11. Lcs Angeles: University of 
Southern California, 1954. 

48. Guilford, J.P., Kettner, N.W., & Christensen, P.R. The relation of certain think- 
ing factors to training criteria in the U.S. Coast Guard academy. Rep. Psychol. 
Lam, No. 13. Los Angeles: University of Southern California, 1955. 

49. Guilford, J.P., Kettner, N.W., & Christensen, P.R. A factor -analytic investiga- 
tion of the factor called general reasoning. Rep. Psychol. Lab., No. 14. Los 
Angeles: University of Southern California, 1955. 

50. Guilford, J. P. , Wilson, R.C., & Christensen, P.R. A factor -analytic study of 
creative thinking. II. Administration of tests and analysis of results. Rep. 
Psychol. Lab., No. 8. Los Angeles: University of Southern California, 1952. 

51. Hadamard, J. The psychology of invention in the mathematical field. Princeton, 

N.J. : Princeton University Press, 1945. ™"~ 

52. Haegen, G. Vander. Les essais de representation mathematique de la brevetabilite 
Revue de l’uniyersite de Bruxelles, 1934, 29, 336-357. 

53. Harris, T.M. The use of projective techniques in industrial selection. 1947 
invitational conference on testing problems. American Council on Education, 1948 . 

54. Hertzka, A. F. , Guilford, J.P., Christensen, P.R., & Berger, R.M. A factor - 
analytic study of evaluative abilities. Educ. psychol. Measmt, 1954, ,14, ,581-597.. 

55. Hills, J.R. The relationship between certain factor -analyzed abilities and success 
in college mathematics. Rep. Psychol. Lab., No. 15. Los Angeles: University 
of Southern California, 1955. 

56. Hutchinson, E.D. How to think creatively . New York: Abingdon -Cokesbury, 1949. 

57. Hughes, E.C. Psychology: science and/or profession. Amer. Psychologist, 1952, 

7, 441-443. 5 — 



58. Johnson, D.M. The psychology of thought and judgment . New York: Harper, 1955. 



59. Kierkegaard, S. The concept of dread. In R. May, The meaning of anxiety. New 
York: Ronald Press, 1950. 

$ 

60. Knapp, R.H. NoninteUective determinants of scientific interests. Middletown, x 
Conn. : Wesleyan Univeristy. 

61. Knapp, R.H. The non -intellective determinants of scientific interest, unpublished 
ONR Report, 1952. 

62. Knapp, R.H. , & Associates. A factor analysis of Thorndike’s ratings of men of 
genius. Wesleyan University Alumnus, June, 1952. 

63. Knapp, R.H., & Goodrich, H.B. Origins of American scientists . Chicago: Univer- 
sity of Chicago Press, 1952. 

64. Knapp, R.H. , & Greenbaum, J.J. The younger American scholar . Chicago: Uni- 
versity of Chicago Press, 1954. 

65. Kroeber, A.L. Configurations of culture growth . Berkeley: University of Califor- 
nia Press, 1944. 

66. Jones, L.V. Primary abilities in the Stanford Binet age 13. J. gen. Psychol., 1954, 
84, 125-147. 

67. Lieberman, S. & Meltzer, L. The attitudes and activities of physiologists . Ann 
Arbor: Institute for social research, University of Michigan, 1954, 2. 

68. Luchins, A.S. The einstellung test of rigidity: its relation to concreteness of 
thinking. J. consult. Psychol. , 1951, U5, 303-310. 

69. MacCurdy, Robert D. Characteristics of superior science students and some 
factors that were found in their backgrounds. Ph.D. dissertation, Boston- Univer- 
sity, 1954. 

70. Mendell, M.W. Measuring originality in the physical sciences. Bduc. psychol. 
Measmt, 1950, 10, 380-385. 

71. Marvick, D. Career perspectives in a bureaucratic setting. Michigan Govern- 
mental Studies, No. 27* Ann Arbor: University of Michigan Press, 1954. 

72. Maslow, S.H. Problem -centering vs. means centering in science. Phil. ofSci., 
1946, 13, 326-331. 

73. Maslow, S.H. Self actualizing people: a picture of psychological health. Person- 
ality Symposium. 1950, 11-34. 



264 







74. McClelland, D.C. Some social consequences of achievement motivation. In Jones, 
M.R. (Ed.), Nebraska symposium on motivation. Lincoln: University of Nebraska 
Press, 1955. 

75. McClelland, D.C. (Ed. ) Studies in motivation. New York: Appleton -Century - 
Crofts, 1955. 

76. McClelland, D.C. The psychology of mental content reconsidered. Psychol. Rev., 
1955, 62, 297-302. 

77. McClelland, D.C., Atkinson, J. W. , Clark, R.A. , & Lowell, E.L*. The achieve- 
ment motive . New York: Appleton -Century -Crofts, 1953. 

78. McClelland, D.C., Rindlisbacher, A., & DeCharms, R. Religious and other 
sources of par ental attitudes toward independence training. In D. A. McClelland 
(Ed.), Studies in motivation. New York: Appleton -Century-Crofts, 1955. 

79. MacCurdy, Robert D. Characteristics of superior science students and some 

. factors that were found in their backgrounds. Ph.D. dissertation, Boston Univer- 
sity, 1954. 

80. Merton, R.K. Social theory and social structure. Glencoe, Illinois: Free Press, 

1949. ' *“ 



81. Miller, J.G. Discrimination without awareness . Amer. J. Psychol., 1939, 52, 

562-578. ~ 

82. Miller, J.G. The role of motivation in learning without awareness . Amer. J. 
Psychol. , 1940, 53, 229-239. 

83. Miller, J.G. Unconsciousness . New York: Wiley, 1942. 

♦ 

84. Mooney, R.L. A preliminary listing of indices of creative behavior. Bureau of 
Educational Research, Columbus: Ohio State University, 1953. 

85. Mooney, R.L. Report on beginning research on the defini tion and ' /elopment of 
creative behavior. Bureau of Educational Research. Columbus: Ohio State Univer- 
sity, 1954. 

86. Ogburn, W.F., & Thomas, D. Are inventions inevitable? Political Science Quar- 
terly, 1922, 37, 38-98. 

87. Olson, Fred, etal. The nature of creative thinking. A monograph of a symposium 
at Skytop, Penna. , sponsored by Industrial Research Institute, Inc. , 1952. 

88. Osborn, Alex F. Applied imagination. New York: Charles Scribner’s Sons# : 1953. . 

89. Osborn, Alex F. Your creative power. New York: Charles Scribner’s Sons, 1950. 



90. Parsons, T. A sociologist looks at the legal profession. Essays on sociological 
theory (rev, ed. ). Glencoe, Illinois: Free Press, 1954. 

91. Parsons, T. Essays in sociological theory: pure and applied . Glencoe, Illinois: 
Free Press, 1949. 

92. Parsons, T., & Shils, E. Toward a general theory of action. Cambridge: Harvard 
University Press, 1951. 

93. Patrick, C. What is creative thinking? New York: Philosophical Library, 1955. 

94. Rees, H.J. , & Israel, E.E. Establishment and operation of mental sets. Psychol. 

Monogr. , 1935, 46, No. 6. ~~ 

95. Rivise, Charles W., & Caesar, A.D. Patentability and validity , Charlottesville, 
Virginia: Michie Company, 1936. 

96. Roe, Anne. A psychological study of eminent biologists . Psychol. Monogr., 1953, 
65, No. 14. 



97. Roe, Anne. A psychological study of physical scientists. Psychol. Monogr., 1951, 
43, 121-239. 

98. Roe, Anne. A psychological study of eminent psychologists and anthropologists and 

a comparison with biological and physical scientists. Psychol. Monogr., 1953, 67, 
No. 2 (Whole No. 352). ~~ 

99. Rogers, C.R. Toward a theory of creativity. University of Chicago, 1955. 

100. Royce, J. The psychology of invention . Psychol. Rev. , 1898, 5, 113-144. 

101. Russell, David H. Children’s thinking . Chaps. 11 and 13. New York: Ginn & 
Company, 1956. 

102. Saunders, David R. Item analysis of interaction variance to produce a criterion 
key for item pairs. ETS research bulletin (in preparation). 

iu3. Saunders, David R. Measured personality differences between engineers employed 
in several functional types of work. ETS Research Memorandum 55-12, Sept. , 

1955, (Abstracted in Amer. Psychologist, 1955, 10, 430-431.) 

104. Saunders, David R. Use of an objective method to determining engineering job 
families that will apply in several companies. ETS Research Bulletin 54-26, 

Sept., 1954. 

105. Shepard, H. A. Field studies in the organization and management of research 
(Progress report, hectograph). Cambridge: School of Industrial Management, 
Massachusetts Institute of Technology, 1954. 



266 



106. Shepard, H.A. Some social attributes of industrial research- and development 
groups (hectograph). Cambridge: School of Industrial Management, Massachu- 
setts Institute of Technology, 1954. 

107. Shockley, W. On the statistics of individual variations of productivity in research 

laboratories. Lecture before the Operations Research Society of America, Wash- 
ington, D.C., Nov., 1954. ■■Hr, 

108. Social Science Research Council. Summary report of committee on identification 
of talents; New York: 1957. 

109. Stein, M.I. Creativity and culture . J. Psychol., 1953, 36 , 311-322. 

i 

110. Stein, M.I. On the role of the industrial research chemist and its relationship 
to the problem of creativity. 

111. Stern. G., Stein, M.I. , & Bloom, B. Methods in personality assessment. 

Glencoe, Illinois: Free Press, 1955. 

112. Stewart, G.W. An attitude toward the growth of ideas . Northwest Missouri State 
Teachers College Studies, 1940. 

113. Stewart, Naomi. Creativity: a literature survey. ETS Research Memorandum 
53-58, 1953. 

114. Stringham, Emerson J.D. Outline of patent law and guide to digests, Madison, 
Wisconsin: Pacot Publications, 1937. 

115. Strong, E.K., Jr. Manual for vocational interest blank for men . Stanford, Cal- 
ifornia: Stanford University Press, 1945. 

116. Strong, E.K., Jr. Vocational interests of men and women . Stanford, California: 
Stanford University Press, 1943. 

117. Taylor, C.W. A factorial study of fluency in writing. Psychometrika, Dec. 1947, 

12, No. 4, 239-262. ” 

118. Terman, L.M. Scientists and nonscientists in a group of 800 gifted men. Psychol. 
Monogr., 1954, 68, No. 7 (Whole No. 378). 

119. Terman, L.M. , & Oden, M. H. The gifted child grows up. Stanford, California: 

Stanford University Press, 1947. ~~ 

120. Thelen, H.E. Experimental research toward a theory of instruction: J, educ. 
Res. , 1951, 45, 89-136. (Bibliog.) 



121. Thurstone, L.L. Creative talent . Psychometric Laboratory, University of 
Chicago, 1950, No. 61. 



122. Thorndike, E.L. The origin of superior men . Scientific Monthly , 1943, 56, 

424-433. 

123. Thorndike, E.L., & Rock, R.T., Jr. Learning without awareness of what is 
being learned or intent to learn it. J. exp. Psychol., 1934, 17, 1-19. 

124. Union College Studies in Character Research, Developing creativity in children’s 
use of imagination, 1953, JL, No. 3. 

125. Walker, Donald E. Creativity and test behaviors ft a chemists and mathematicians. 
Fii.D. dissertation, University of Chicago, 1955. 

126. Weber, M. The protestant ethic (translated by Talcott Parsons). New York: 
Scribner’s, 1930. 

127. Wertheimer, M. Productive thinking . New York: Harper, 1945. 

128. Whyte, William H., Jr. The fallacies of personality testing. Fortune, 1954, 50, 

(3), 117-121. 

129. Wilson, R.C., Guilford, J.P., & Christensen, P.R. The measurement of individual 
differences in originality. Psychol. Bull. , 1953, 50, 362-370. 

t 

130. Wilson, R.C., Guilford, J.P., Christensen, P.R., & Lewis, D.J. A factor -analytic 
study of creative thinking abilities . Psychometrika, 1954, 19, 297-311. 

131. Wrightstone, J. W Measuring the social climate of a classroom. J. educ. Res., 
,1951,(44, 341-351, (Bibliog.) 




268 



